You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2017/07/21 17:21:58 UTC
[JENKINS] Lucene-Solr-7.x-MacOSX (64bit/jdk1.8.0) - Build # 60 -
Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-MacOSX/60/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseParallelGC
1 tests failed.
FAILED: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test
Error Message:
Could not find collection:collection2
Stack Trace:
java.lang.AssertionError: Could not find collection:collection2
at __randomizedtesting.SeedInfo.seed([B95362923D2FBC6E:31075D4893D3D196]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.junit.Assert.assertNotNull(Assert.java:526)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:155)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:140)
at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:135)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:907)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrClient(FullSolrCloudDistribCmdsTest.java:612)
at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test(FullSolrCloudDistribCmdsTest.java:152)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 11988 lines...]
[junit4] Suite: org.apache.solr.cloud.FullSolrCloudDistribCmdsTest
[junit4] 2> 1527742 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[B95362923D2FBC6E]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/init-core-data-001
[junit4] 2> 1527742 WARN (SUITE-FullSolrCloudDistribCmdsTest-seed#[B95362923D2FBC6E]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=88 numCloses=88
[junit4] 2> 1527743 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[B95362923D2FBC6E]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 1527747 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[B95362923D2FBC6E]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776) w/ MAC_OS_X supressed clientAuth
[junit4] 2> 1527747 INFO (SUITE-FullSolrCloudDistribCmdsTest-seed#[B95362923D2FBC6E]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 1527749 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1527750 INFO (Thread-2889) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1527750 INFO (Thread-2889) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1527753 ERROR (Thread-2889) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1527861 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.ZkTestServer start zk server on port:60915
[junit4] 2> 1527932 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1527935 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test-files/solr/collection1/conf/schema15.xml to /configs/conf1/schema.xml
[junit4] 2> 1527938 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1527940 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1527942 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1527944 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1527947 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1527949 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1527952 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1527954 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1527957 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.AbstractZkTestCase put /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1527960 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 1528307 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1528308 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3ed6533{/,null,AVAILABLE}
[junit4] 2> 1528309 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@36d4dbfc{HTTP/1.1,[http/1.1]}{127.0.0.1:60918}
[junit4] 2> 1528309 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.e.j.s.Server Started @1531962ms
[junit4] 2> 1528310 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/tempDir-001/control/data, replicaType=NRT, hostContext=/, hostPort=60918, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/control-001/cores}
[junit4] 2> 1528310 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1528310 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 1528310 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1528310 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 1528310 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-07-21T16:42:20.805Z
[junit4] 2> 1528327 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1528327 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/control-001/solr.xml
[junit4] 2> 1528338 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1528342 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60915/solr
[junit4] 2> 1528535 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60918_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1528536 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60918_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:60918_
[junit4] 2> 1528538 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60918_ ] o.a.s.c.Overseer Overseer (id=98346948378427396-127.0.0.1:60918_-n_0000000000) starting
[junit4] 2> 1528550 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60918_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60918_
[junit4] 2> 1528552 INFO (zkCallback-2339-thread-1-processing-n:127.0.0.1:60918_) [n:127.0.0.1:60918_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1528679 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60918_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49
[junit4] 2> 1528685 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60918_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49
[junit4] 2> 1528685 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60918_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49
[junit4] 2> 1528686 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60918_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/control-001/cores
[junit4] 2> 1528718 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1528725 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:60915/solr ready
[junit4] 2> 1528735 INFO (qtp1051115645-13528) [n:127.0.0.1:60918_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:60918_&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1528738 INFO (OverseerThreadFactory-5967-thread-1-processing-n:127.0.0.1:60918_) [n:127.0.0.1:60918_ ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 1528852 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1528852 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1528964 INFO (zkCallback-2339-thread-1-processing-n:127.0.0.1:60918_) [n:127.0.0.1:60918_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1529871 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 1529881 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 1529952 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1529972 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 1529972 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49
[junit4] 2> 1529972 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1529972 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 1529974 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=28, maxMergeAtOnceExplicit=41, maxMergedSegmentMB=65.666015625, floorSegmentMB=1.30859375, forceMergeDeletesPctAllowed=27.527609988890774, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 1529976 WARN (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1530023 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1530023 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1530024 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1530024 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1530025 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=42, maxMergedSegmentMB=26.3154296875, floorSegmentMB=1.24609375, forceMergeDeletesPctAllowed=17.09449073833258, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.37504926339891076
[junit4] 2> 1530025 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@9e49ca2[control_collection_shard1_replica_n1] main]
[junit4] 2> 1530027 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1530027 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1530028 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1530028 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1573551176441397248
[junit4] 2> 1530028 INFO (searcherExecutor-5970-thread-1-processing-n:127.0.0.1:60918_ x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@9e49ca2[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1530034 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1530034 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1530034 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:60918/control_collection_shard1_replica_n1/
[junit4] 2> 1530034 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1530034 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:60918/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 1530034 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1530040 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:60918/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 1530151 INFO (zkCallback-2339-thread-1-processing-n:127.0.0.1:60918_) [n:127.0.0.1:60918_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1530200 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1530202 INFO (qtp1051115645-13529) [n:127.0.0.1:60918_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1350
[junit4] 2> 1530206 INFO (qtp1051115645-13528) [n:127.0.0.1:60918_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1530307 INFO (zkCallback-2339-thread-1-processing-n:127.0.0.1:60918_) [n:127.0.0.1:60918_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1530746 INFO (OverseerCollectionConfigSetProcessor-98346948378427396-127.0.0.1:60918_-n_0000000000) [n:127.0.0.1:60918_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1531215 INFO (qtp1051115645-13528) [n:127.0.0.1:60918_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:60918_&wt=javabin&version=2} status=0 QTime=2480
[junit4] 2> 1531223 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1531225 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:60915/solr ready
[junit4] 2> 1531225 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 1531226 INFO (qtp1051115645-13533) [n:127.0.0.1:60918_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=3&createNodeSet=&stateFormat=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1531229 INFO (OverseerThreadFactory-5967-thread-2-processing-n:127.0.0.1:60918_) [n:127.0.0.1:60918_ ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 1531230 WARN (OverseerThreadFactory-5967-thread-2-processing-n:127.0.0.1:60918_) [n:127.0.0.1:60918_ ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 1531440 INFO (qtp1051115645-13533) [n:127.0.0.1:60918_ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1531440 INFO (qtp1051115645-13533) [n:127.0.0.1:60918_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=3&createNodeSet=&stateFormat=1&wt=javabin&version=2} status=0 QTime=214
[junit4] 2> 1531946 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-1-001 of type NRT
[junit4] 2> 1531947 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1531948 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3a500999{/,null,AVAILABLE}
[junit4] 2> 1531949 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@79b871f8{HTTP/1.1,[http/1.1]}{127.0.0.1:60928}
[junit4] 2> 1531949 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.e.j.s.Server Started @1535601ms
[junit4] 2> 1531949 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/tempDir-001/jetty1, solrconfig=solrconfig.xml, hostContext=/, hostPort=60928, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-1-001/cores}
[junit4] 2> 1531950 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1531950 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 1531950 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1531950 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 1531951 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-07-21T16:42:24.446Z
[junit4] 2> 1531954 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1531954 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-1-001/solr.xml
[junit4] 2> 1531961 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1531966 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60915/solr
[junit4] 2> 1531983 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60928_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1531988 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60928_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1531991 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60928_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60928_
[junit4] 2> 1531994 INFO (zkCallback-2346-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1531994 INFO (zkCallback-2339-thread-1-processing-n:127.0.0.1:60918_) [n:127.0.0.1:60918_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1531994 INFO (zkCallback-2351-thread-1-processing-n:127.0.0.1:60928_) [n:127.0.0.1:60928_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1532106 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60928_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49
[junit4] 2> 1532111 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60928_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49
[junit4] 2> 1532111 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60928_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49
[junit4] 2> 1532113 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60928_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-1-001/cores
[junit4] 2> 1532145 INFO (qtp608269624-13569) [n:127.0.0.1:60928_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:60928_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1532148 INFO (OverseerThreadFactory-5967-thread-3-processing-n:127.0.0.1:60918_) [n:127.0.0.1:60918_ ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:60928_ for creating new replica
[junit4] 2> 1532148 INFO (OverseerCollectionConfigSetProcessor-98346948378427396-127.0.0.1:60918_-n_0000000000) [n:127.0.0.1:60918_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1532152 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n1&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1532154 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1533620 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 1533679 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.s.IndexSchema [collection1_shard2_replica_n1] Schema name=test
[junit4] 2> 1533824 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1533849 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n1' using configuration from collection collection1, trusted=true
[junit4] 2> 1533850 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n1' (registry 'solr.core.collection1.shard2.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49
[junit4] 2> 1533850 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1533850 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.c.SolrCore [[collection1_shard2_replica_n1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-1-001/cores/collection1_shard2_replica_n1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/../../../../../../../../../Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-1-001/cores/collection1_shard2_replica_n1/data/]
[junit4] 2> 1533852 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=28, maxMergeAtOnceExplicit=41, maxMergedSegmentMB=65.666015625, floorSegmentMB=1.30859375, forceMergeDeletesPctAllowed=27.527609988890774, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 1533855 WARN (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1533978 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1533978 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1533979 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1533979 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1533981 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=42, maxMergedSegmentMB=26.3154296875, floorSegmentMB=1.24609375, forceMergeDeletesPctAllowed=17.09449073833258, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.37504926339891076
[junit4] 2> 1533982 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@28043309[collection1_shard2_replica_n1] main]
[junit4] 2> 1533989 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1533990 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1533991 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1533991 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1573551180596903936
[junit4] 2> 1533992 INFO (searcherExecutor-5981-thread-1-processing-n:127.0.0.1:60928_ x:collection1_shard2_replica_n1 s:shard2 c:collection1) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.c.SolrCore [collection1_shard2_replica_n1] Registered new searcher Searcher@28043309[collection1_shard2_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1534003 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1534003 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1534003 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:60928/collection1_shard2_replica_n1/
[junit4] 2> 1534003 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1534003 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:60928/collection1_shard2_replica_n1/ has no replicas
[junit4] 2> 1534003 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1534025 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:60928/collection1_shard2_replica_n1/ shard2
[junit4] 2> 1534150 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1534154 INFO (qtp608269624-13570) [n:127.0.0.1:60928_ c:collection1 s:shard2 x:collection1_shard2_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n1&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2001
[junit4] 2> 1534157 INFO (qtp608269624-13569) [n:127.0.0.1:60928_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:60928_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2} status=0 QTime=2012
[junit4] 2> 1534158 INFO (OverseerCollectionConfigSetProcessor-98346948378427396-127.0.0.1:60918_-n_0000000000) [n:127.0.0.1:60918_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1534855 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-2-001 of type NRT
[junit4] 2> 1534855 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1534857 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1454a602{/,null,AVAILABLE}
[junit4] 2> 1534858 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@91e53b5{HTTP/1.1,[http/1.1]}{127.0.0.1:60934}
[junit4] 2> 1534858 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.e.j.s.Server Started @1538510ms
[junit4] 2> 1534858 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/tempDir-001/jetty2, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=60934, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-2-001/cores}
[junit4] 2> 1534858 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1534859 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 1534859 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1534859 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 1534859 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-07-21T16:42:27.354Z
[junit4] 2> 1534862 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1534862 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-2-001/solr.xml
[junit4] 2> 1534869 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1534874 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60915/solr
[junit4] 2> 1534902 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60934_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1534905 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60934_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1534909 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60934_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60934_
[junit4] 2> 1534911 INFO (zkCallback-2346-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1534911 INFO (zkCallback-2351-thread-1-processing-n:127.0.0.1:60928_) [n:127.0.0.1:60928_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1534912 INFO (zkCallback-2339-thread-1-processing-n:127.0.0.1:60918_) [n:127.0.0.1:60918_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1534912 INFO (zkCallback-2357-thread-1-processing-n:127.0.0.1:60934_) [n:127.0.0.1:60934_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 1535004 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60934_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49
[junit4] 2> 1535009 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60934_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49
[junit4] 2> 1535009 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60934_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49
[junit4] 2> 1535011 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60934_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-2-001/cores
[junit4] 2> 1535044 INFO (qtp608269624-13572) [n:127.0.0.1:60928_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:60934_&action=ADDREPLICA&collection=collection1&shard=shard3&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1535047 INFO (OverseerThreadFactory-5967-thread-4-processing-n:127.0.0.1:60918_) [n:127.0.0.1:60918_ ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:60934_ for creating new replica
[junit4] 2> 1535053 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard3_replica_n3&action=CREATE&collection=collection1&shard=shard3&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1535053 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1536078 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 1536103 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.s.IndexSchema [collection1_shard3_replica_n3] Schema name=test
[junit4] 2> 1536191 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1536228 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard3_replica_n3' using configuration from collection collection1, trusted=true
[junit4] 2> 1536229 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard3.replica_n3' (registry 'solr.core.collection1.shard3.replica_n3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49
[junit4] 2> 1536229 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1536229 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.c.SolrCore [[collection1_shard3_replica_n3] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-2-001/cores/collection1_shard3_replica_n3], dataDir=[/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-2-001/cores/collection1_shard3_replica_n3/data/]
[junit4] 2> 1536232 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=28, maxMergeAtOnceExplicit=41, maxMergedSegmentMB=65.666015625, floorSegmentMB=1.30859375, forceMergeDeletesPctAllowed=27.527609988890774, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 1536271 WARN (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1536344 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1536344 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1536346 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1536346 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1536362 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=42, maxMergedSegmentMB=26.3154296875, floorSegmentMB=1.24609375, forceMergeDeletesPctAllowed=17.09449073833258, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.37504926339891076
[junit4] 2> 1536363 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.s.SolrIndexSearcher Opening [Searcher@3fb494fc[collection1_shard3_replica_n3] main]
[junit4] 2> 1536364 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1536365 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1536366 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1536366 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1573551183087271936
[junit4] 2> 1536367 INFO (searcherExecutor-5992-thread-1-processing-n:127.0.0.1:60934_ x:collection1_shard3_replica_n3 s:shard3 c:collection1) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.c.SolrCore [collection1_shard3_replica_n3] Registered new searcher Searcher@3fb494fc[collection1_shard3_replica_n3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1536375 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1536375 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1536376 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:60934/collection1_shard3_replica_n3/
[junit4] 2> 1536376 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1536376 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.c.SyncStrategy http://127.0.0.1:60934/collection1_shard3_replica_n3/ has no replicas
[junit4] 2> 1536376 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1536385 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:60934/collection1_shard3_replica_n3/ shard3
[junit4] 2> 1536497 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1536500 INFO (qtp867811010-13598) [n:127.0.0.1:60934_ c:collection1 s:shard3 x:collection1_shard3_replica_n3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard3_replica_n3&action=CREATE&collection=collection1&shard=shard3&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1447
[junit4] 2> 1536502 INFO (qtp608269624-13572) [n:127.0.0.1:60928_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:60934_&action=ADDREPLICA&collection=collection1&shard=shard3&type=NRT&wt=javabin&version=2} status=0 QTime=1458
[junit4] 2> 1536965 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-3-001 of type NRT
[junit4] 2> 1536966 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1536967 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@360cc93{/,null,AVAILABLE}
[junit4] 2> 1536967 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@3482f2d4{HTTP/1.1,[http/1.1]}{127.0.0.1:60939}
[junit4] 2> 1536968 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.e.j.s.Server Started @1540620ms
[junit4] 2> 1536968 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/tempDir-001/jetty3, replicaType=NRT, solrconfig=solrconfig.xml, hostContext=/, hostPort=60939, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-3-001/cores}
[junit4] 2> 1536968 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1536969 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 1536969 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1536969 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 1536969 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-07-21T16:42:29.464Z
[junit4] 2> 1536973 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1536973 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-3-001/solr.xml
[junit4] 2> 1536981 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1536986 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60915/solr
[junit4] 2> 1537011 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60939_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 1537017 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60939_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1537025 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60939_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60939_
[junit4] 2> 1537029 INFO (zkCallback-2363-thread-1-processing-n:127.0.0.1:60939_) [n:127.0.0.1:60939_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1537029 INFO (zkCallback-2346-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1537029 INFO (zkCallback-2351-thread-1-processing-n:127.0.0.1:60928_) [n:127.0.0.1:60928_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1537029 INFO (zkCallback-2339-thread-1-processing-n:127.0.0.1:60918_) [n:127.0.0.1:60918_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1537032 INFO (zkCallback-2357-thread-1-processing-n:127.0.0.1:60934_) [n:127.0.0.1:60934_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 1537058 INFO (OverseerCollectionConfigSetProcessor-98346948378427396-127.0.0.1:60918_-n_0000000000) [n:127.0.0.1:60918_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1537201 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60939_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49
[junit4] 2> 1537207 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60939_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49
[junit4] 2> 1537207 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60939_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49
[junit4] 2> 1537209 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60939_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-3-001/cores
[junit4] 2> 1537258 INFO (qtp608269624-13573) [n:127.0.0.1:60928_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:60939_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1537261 INFO (OverseerThreadFactory-5967-thread-5-processing-n:127.0.0.1:60918_) [n:127.0.0.1:60918_ ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:60939_ for creating new replica
[junit4] 2> 1537267 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n5&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1537268 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1538320 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 1538335 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.s.IndexSchema [collection1_shard1_replica_n5] Schema name=test
[junit4] 2> 1539670 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1539711 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n5' using configuration from collection collection1, trusted=true
[junit4] 2> 1539712 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_n5' (registry 'solr.core.collection1.shard1.replica_n5') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49
[junit4] 2> 1539712 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1539712 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.c.SolrCore [[collection1_shard1_replica_n5] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-3-001/cores/collection1_shard1_replica_n5], dataDir=[/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-3-001/cores/collection1_shard1_replica_n5/data/]
[junit4] 2> 1539714 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=28, maxMergeAtOnceExplicit=41, maxMergedSegmentMB=65.666015625, floorSegmentMB=1.30859375, forceMergeDeletesPctAllowed=27.527609988890774, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 1539723 WARN (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1539798 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1539798 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1539799 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1539799 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1539805 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=42, maxMergedSegmentMB=26.3154296875, floorSegmentMB=1.24609375, forceMergeDeletesPctAllowed=17.09449073833258, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.37504926339891076
[junit4] 2> 1539805 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.s.SolrIndexSearcher Opening [Searcher@436981f4[collection1_shard1_replica_n5] main]
[junit4] 2> 1539807 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1539808 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1539809 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1539809 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1573551186697519104
[junit4] 2> 1539810 INFO (searcherExecutor-6003-thread-1-processing-n:127.0.0.1:60939_ x:collection1_shard1_replica_n5 s:shard1 c:collection1) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.c.SolrCore [collection1_shard1_replica_n5] Registered new searcher Searcher@436981f4[collection1_shard1_replica_n5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1539818 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1539818 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1539818 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:60939/collection1_shard1_replica_n5/
[junit4] 2> 1539818 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1539818 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.c.SyncStrategy http://127.0.0.1:60939/collection1_shard1_replica_n5/ has no replicas
[junit4] 2> 1539818 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1539825 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:60939/collection1_shard1_replica_n5/ shard1
[junit4] 2> 1539989 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1539991 INFO (qtp666491162-13627) [n:127.0.0.1:60939_ c:collection1 s:shard1 x:collection1_shard1_replica_n5] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_n5&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2724
[junit4] 2> 1539995 INFO (qtp608269624-13573) [n:127.0.0.1:60928_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:60939_&action=ADDREPLICA&collection=collection1&shard=shard1&type=NRT&wt=javabin&version=2} status=0 QTime=2737
[junit4] 2> 1540427 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 4 in directory /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-4-001 of type NRT
[junit4] 2> 1540427 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1540450 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3946f90{/,null,AVAILABLE}
[junit4] 2> 1540452 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@5793c768{HTTP/1.1,[http/1.1]}{127.0.0.1:60944}
[junit4] 2> 1540453 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.e.j.s.Server Started @1544105ms
[junit4] 2> 1540453 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/tempDir-001/jetty4, solrconfig=solrconfig.xml, hostContext=/, hostPort=60944, coreRootDirectory=/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-4-001/cores}
[junit4] 2> 1540454 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1540455 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.1.0
[junit4] 2> 1540455 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1540455 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 1540455 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-07-21T16:42:32.950Z
[junit4] 2> 1540460 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1540460 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-4-001/solr.xml
[junit4] 2> 1540468 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1540477 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60915/solr
[junit4] 2> 1540495 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60944_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (4)
[junit4] 2> 1540501 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60944_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1540505 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60944_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60944_
[junit4] 2> 1540508 INFO (zkCallback-2363-thread-1-processing-n:127.0.0.1:60939_) [n:127.0.0.1:60939_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1540508 INFO (zkCallback-2351-thread-1-processing-n:127.0.0.1:60928_) [n:127.0.0.1:60928_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1540508 INFO (zkCallback-2369-thread-1-processing-n:127.0.0.1:60944_) [n:127.0.0.1:60944_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1540508 INFO (zkCallback-2357-thread-1-processing-n:127.0.0.1:60934_) [n:127.0.0.1:60934_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1540508 INFO (zkCallback-2346-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1540508 INFO (zkCallback-2339-thread-1-processing-n:127.0.0.1:60918_) [n:127.0.0.1:60918_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (4) -> (5)
[junit4] 2> 1540629 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60944_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49
[junit4] 2> 1540634 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60944_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49
[junit4] 2> 1540635 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60944_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49
[junit4] 2> 1540636 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [n:127.0.0.1:60944_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-4-001/cores
[junit4] 2> 1540683 INFO (qtp608269624-13574) [n:127.0.0.1:60928_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:60944_&action=ADDREPLICA&collection=collection1&shard=shard2&type=NRT&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1540686 INFO (OverseerThreadFactory-5967-thread-5-processing-n:127.0.0.1:60918_) [n:127.0.0.1:60918_ ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:60944_ for creating new replica
[junit4] 2> 1540687 INFO (OverseerCollectionConfigSetProcessor-98346948378427396-127.0.0.1:60918_-n_0000000000) [n:127.0.0.1:60918_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000008 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1540692 INFO (qtp2144338523-13656) [n:127.0.0.1:60944_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n7&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1540693 INFO (qtp2144338523-13656) [n:127.0.0.1:60944_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1541719 INFO (qtp2144338523-13656) [n:127.0.0.1:60944_ c:collection1 s:shard2 x:collection1_shard2_replica_n7] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 1541731 INFO (qtp2144338523-13656) [n:127.0.0.1:60944_ c:collection1 s:shard2 x:collection1_shard2_replica_n7] o.a.s.s.IndexSchema [collection1_shard2_replica_n7] Schema name=test
[junit4] 2> 1541810 INFO (qtp2144338523-13656) [n:127.0.0.1:60944_ c:collection1 s:shard2 x:collection1_shard2_replica_n7] o.a.s.s.IndexSchema Loaded schema test/1.6 with uniqueid field id
[junit4] 2> 1541836 INFO (qtp2144338523-13656) [n:127.0.0.1:60944_ c:collection1 s:shard2 x:collection1_shard2_replica_n7] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n7' using configuration from collection collection1, trusted=true
[junit4] 2> 1541836 INFO (qtp2144338523-13656) [n:127.0.0.1:60944_ c:collection1 s:shard2 x:collection1_shard2_replica_n7] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_n7' (registry 'solr.core.collection1.shard2.replica_n7') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@5460ae49
[junit4] 2> 1541837 INFO (qtp2144338523-13656) [n:127.0.0.1:60944_ c:collection1 s:shard2 x:collection1_shard2_replica_n7] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1541837 INFO (qtp2144338523-13656) [n:127.0.0.1:60944_ c:collection1 s:shard2 x:collection1_shard2_replica_n7] o.a.s.c.SolrCore [[collection1_shard2_replica_n7] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-4-001/cores/collection1_shard2_replica_n7], dataDir=[/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001/shard-4-001/cores/collection1_shard2_replica_n7/data/]
[junit4] 2> 1541839 INFO (qtp2144338523-13656) [n:127.0.0.1:60944_ c:collection1 s:shard2 x:collection1_shard2_replica_n7] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=28, maxMergeAtOnceExplicit=41, maxMergedSegmentMB=65.666015625, floorSegmentMB=1.30859375, forceMergeDeletesPctAllowed=27.527609988890774, segmentsPerTier=13.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
[junit4] 2> 1541855 WARN (qtp2144338523-13656) [n:127.0.0.1:60944_ c:collection1 s:shard2 x:collection1_shard2_replica_n7] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1541921 INFO (qtp2144338523-13656) [n:127.0.0.1:60944_ c:collection1 s:shard2 x:collection1_shard2_replica_n7] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1541921 INFO (qtp2144338523-13656) [n:127.0.0.1:60944_ c:collection1 s:shard2 x:collection1_shard2_replica_n7] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1541923 INFO (qtp2144338523-13656) [n:127.0.0.1:60944_ c:collection1 s:shard2 x:collection1_shard2_replica_n7] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1541923 INFO (qtp2144338523-13656) [n:127.0.0.1:60944_ c:collection1 s:shard2 x:collection1_shard2_replica_n7] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1541924 INFO (qtp2144338523-13656) [n:127.0.0.1:60944_ c:collection1 s:shard2 x:collection1_shard2_replica_n7] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=39, maxMergeAtOnceExplicit=42, maxMergedSegmentMB=26.3154296875, floorSegmentMB=1.24609375, forceMergeDeletesPctAllowed=17.09449073833258, segmentsPerTier=34.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.37504926339891076
[junit4] 2> 1541925 INFO (qtp2144338523-13656) [n:127.0.0.1:60944_ c:collection1 s:shard2 x:collection1_shard2_replica_n7] o.a.s.s.SolrIndexSearcher Opening [Searcher@7f0bdf88[collection1_shard2_replica_n7] main]
[junit4] 2> 1541926 INFO (qtp2144338523-13656) [n:127.0.0.1:60944_ c:collection1 s:shard2 x:collection1_shard2_replica_n7] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1541927 INFO (qtp2144338523-13656) [n:127.0.0.1:60944_ c:collection1 s:shard2 x:collection1_shard2_replica_n7] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1541927 INFO (qtp2144338523-13656) [n:127.0.0.1:60944_ c:collection1 s:shard2 x:collection1_shard2_replica_n7] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1541927 INFO (qtp2144338523-13656) [n:127.0.0.1:60944_ c:collection1 s:shard2 x:collection1_shard2_replica_n7] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1573551188918403072
[junit4] 2> 1541928 INFO (searcherExecutor-6014-thread-1-processing-n:127.0.0.1:60944_ x:collection1_shard2_replica_n7 s:shard2 c:collection1) [n:127.0.0.1:60944_ c:collection1 s:shard2 x:collection1_shard2_replica_n7] o.a.s.c.SolrCore [collection1_shard2_replica_n7] Registered new searcher Searcher@7f0bdf88[collection1_shard2_replica_n7] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1541931 INFO (qtp2144338523-13656) [n:127.0.0.1:60944_ c:collection1 s:shard2 x:collection1_shard2_replica_n7] o.a.s.c.ZkController Core needs to recover:collection1_shard2_replica_n7
[junit4] 2> 1541932 INFO (updateExecutor-2366-thread-1-processing-n:127.0.0.1:60944_ x:collection1_shard2_replica_n7 s:shard2 c:collection1) [n:127.0.0.1:60944_ c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 1541932 INFO (recoveryExecutor-2367-thread-1-processing-n:127.0.0.1:60944_ x:collection1_shard2_replica_n7 s:shard2 c:collection1 r:core_node8) [n:127.0.0.1:60944_ c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 1541933 INFO (recoveryExecutor-2367-thread-1-processing-n:127.0.0.1:60944_ x:collection1_shard2_replica_n7 s:shard2 c:collection1 r:core_node8) [n:127.0.0.1:60944_ c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 1541933 INFO (qtp2144338523-13656) [n:127.0.0.1:60944_ c:collection1 s:shard2 x:collection1_shard2_replica_n7] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_n7&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1240
[junit4] 2> 1541933 INFO (recoveryExecutor-2367-thread-1-processing-n:127.0.0.1:60944_ x:collection1_shard2_replica_n7 s:shard2 c:collection1 r:core_node8) [n:127.0.0.1:60944_ c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_n7] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1_shard2_replica_n7]
[jun
[...truncated too long message...]
eContainer Shutting down CoreContainer instance=2010263391
[junit4] 2> 1592665 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1592665 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1592665 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1592666 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1592668 WARN (zkCallback-2376-thread-2-processing-n:127.0.0.1:60950_) [n:127.0.0.1:60950_ ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 1592669 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@2ec20988{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 1592669 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@35256f9a{/,null,UNAVAILABLE}
[junit4] 2> 1592669 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.ChaosMonkey monkey: stop jetty! 60957
[junit4] 2> 1592670 ERROR (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1592671 INFO (TEST-FullSolrCloudDistribCmdsTest.test-seed#[B95362923D2FBC6E]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:60915 60915
[junit4] 2> 1592687 INFO (Thread-2889) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:60915 60915
[junit4] 2> 1592873 WARN (Thread-2889) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 15 /solr/aliases.json
[junit4] 2> 11 /solr/configs/conf1
[junit4] 2> 7 /solr/security.json
[junit4] 2> 6 /solr/collections/collection1/state.json
[junit4] 2> 5 /solr/collections/collection2/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 15 /solr/clusterprops.json
[junit4] 2> 15 /solr/clusterstate.json
[junit4] 2> 2 /solr/collections/collection2/leader_elect/shard5/election/98346948378427413-core_node9-n_0000000000
[junit4] 2> 2 /solr/overseer_elect/election/98346948378427404-127.0.0.1:60934_-n_0000000002
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 15 /solr/live_nodes
[junit4] 2> 15 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=FullSolrCloudDistribCmdsTest -Dtests.method=test -Dtests.seed=B95362923D2FBC6E -Dtests.slow=true -Dtests.locale=ca -Dtests.timezone=Eire -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] FAILURE 65.1s J0 | FullSolrCloudDistribCmdsTest.test <<<
[junit4] > Throwable #1: java.lang.AssertionError: Could not find collection:collection2
[junit4] > at __randomizedtesting.SeedInfo.seed([B95362923D2FBC6E:31075D4893D3D196]:0)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:155)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:140)
[junit4] > at org.apache.solr.cloud.AbstractDistribZkTestBase.waitForRecoveriesToFinish(AbstractDistribZkTestBase.java:135)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.waitForRecoveriesToFinish(AbstractFullDistribZkTestBase.java:907)
[junit4] > at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.testIndexingBatchPerRequestWithHttpSolrClient(FullSolrCloudDistribCmdsTest.java:612)
[junit4] > at org.apache.solr.cloud.FullSolrCloudDistribCmdsTest.test(FullSolrCloudDistribCmdsTest.java:152)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:985)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:960)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.FullSolrCloudDistribCmdsTest_B95362923D2FBC6E-001
[junit4] 2> Jul 21, 2017 4:43:25 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=false): {}, locale=ca, timezone=Eire
[junit4] 2> NOTE: Mac OS X 10.11.6 x86_64/Oracle Corporation 1.8.0_131 (64-bit)/cpus=3,threads=1,free=29358408,total=457703424
[junit4] 2> NOTE: All tests run in this JVM: [TestReqParamsAPI, OutputWriterTest, ReplaceNodeTest, UnloadDistributedZkTest, TestHdfsBackupRestoreCore, TestSolrDeletionPolicy2, TestTrieFacet, CleanupOldIndexTest, TestGroupingSearch, SpatialFilterTest, PrimUtilsTest, NotRequiredUniqueKeyTest, CoreSorterTest, TestRawResponseWriter, ExitableDirectoryReaderTest, TestSearcherReuse, TestSubQueryTransformer, HdfsSyncSliceTest, DateFieldTest, CollectionReloadTest, TestSolrQueryResponse, SolrSlf4jReporterTest, ShufflingReplicaListTransformerTest, TestAnalyzeInfixSuggestions, TestClassicSimilarityFactory, TestPseudoReturnFields, TestCSVResponseWriter, CoreAdminRequestStatusTest, ClusterStateTest, TestCopyFieldCollectionResource, TestWriterPerf, TestConfigSetsAPIZkFailure, InfoHandlerTest, TestEmbeddedSolrServerSchemaAPI, SolrMetricManagerTest, SolrCoreTest, BitVectorTest, ConfigSetsAPITest, TestDocTermOrds, XsltUpdateRequestHandlerTest, TestTolerantSearch, ResponseLogComponentTest, HdfsWriteToMultipleCollectionsTest, TestMiniSolrCloudCluster, ClusterStateUpdateTest, PKIAuthenticationIntegrationTest, ZkControllerTest, ChaosMonkeyNothingIsSafeTest, RecoveryZkTest, PrimitiveFieldTypeTest, TestFieldCacheReopen, TestSort, UniqFieldsUpdateProcessorFactoryTest, SSLMigrationTest, CdcrVersionReplicationTest, RequestLoggingTest, TestImpersonationWithHadoopAuth, AnalysisErrorHandlingTest, PathHierarchyTokenizerFactoryTest, HdfsTlogReplayBufferedWhileIndexingTest, AsyncCallRequestStatusResponseTest, TestCloudJSONFacetJoinDomain, MultiThreadedOCPTest, TestQueryWrapperFilter, MetricUtilsTest, SuggestComponentContextFilterQueryTest, TestGeoJSONResponseWriter, RequestHandlersTest, TestSubQueryTransformerDistrib, URLClassifyProcessorTest, CSVRequestHandlerTest, TestFilteredDocIdSet, TestFunctionQuery, BJQParserTest, SortByFunctionTest, HardAutoCommitTest, TestUtils, FullHLLTest, TestClusterStateMutator, MissingSegmentRecoveryTest, DataDrivenBlockJoinTest, HdfsChaosMonkeySafeLeaderTest, QueryElevationComponentTest, QueryParsingTest, UUIDFieldTest, TestPolicyCloud, TestFieldTypeResource, TestReplicationHandler, RestartWhileUpdatingTest, BlockDirectoryTest, TestClusterProperties, TestConfigOverlay, SolrIndexMetricsTest, NumericFieldsTest, FastVectorHighlighterTest, TestJavabinTupleStreamParser, TestSolrConfigHandler, TestManagedSynonymFilterFactory, TestComplexPhraseLeadingWildcard, IndexBasedSpellCheckerTest, TestStressReorder, TestFieldResource, TestInfoStreamLogging, TestConfigsApi, TestLegacyNumericUtils, BasicAuthIntegrationTest, TestXmlQParserPlugin, OverseerTest, TestUseDocValuesAsStored2, TestDFRSimilarityFactory, CollectionTooManyReplicasTest, AnalysisAfterCoreReloadTest, TestMultiValuedNumericRangeQuery, TestAddFieldRealTimeGet, TestNestedDocsSort, ForceLeaderTest, OverseerCollectionConfigSetProcessorTest, TestStandardQParsers, CollectionsAPISolrJTest, AtomicUpdatesTest, TestSizeLimitedDistributedMap, TestRecoveryHdfs, TestHalfAndHalfDocValues, DistributedFacetPivotSmallTest, RuleEngineTest, RankQueryTest, TestSchemalessBufferedUpdates, TestBlobHandler, TestDistributedSearch, OutOfBoxZkACLAndCredentialsProvidersTest, TestPHPSerializedResponseWriter, TestHdfsCloudBackupRestore, TestConfigSetsAPI, TestFieldCacheSortRandom, HdfsRecoveryZkTest, TestQuerySenderListener, CoreAdminHandlerTest, TestSolrCLIRunExample, GraphQueryTest, MoveReplicaHDFSUlogDirTest, OverseerTaskQueueTest, TestStreamBody, CreateCollectionCleanupTest, DistanceUnitsTest, TestObjectReleaseTracker, TestFileDictionaryLookup, TestPhraseSuggestions, HighlighterMaxOffsetTest, BinaryUpdateRequestHandlerTest, OverseerRolesTest, TestHashPartitioner, DeleteInactiveReplicaTest, TestReplicaProperties, TestNonDefinedSimilarityFactory, DateRangeFieldTest, SegmentsInfoRequestHandlerTest, TestLocalFSCloudBackupRestore, TestUniqueKeyFieldResource, ConjunctionSolrSpellCheckerTest, TestMiniSolrCloudClusterSSL, TestSegmentSorting, ZkCLITest, SimpleFacetsTest, TestScoreJoinQPScore, DistribCursorPagingTest, TestNumericRangeQuery64, TestHttpShardHandlerFactory, TestDistributedStatsComponentCardinality, TestShardHandlerFactory, HdfsDirectoryTest, AddSchemaFieldsUpdateProcessorFactoryTest, TestFaceting, CloneFieldUpdateProcessorFactoryTest, TestSolrCloudSnapshots, TestDynamicFieldResource, SmileWriterTest, TemplateUpdateProcessorTest, SyncSliceTest, ShardRoutingTest, FullSolrCloudDistribCmdsTest]
[junit4] Completed [374/728 (1!)] on J0 in 65.94s, 1 test, 1 failure <<< FAILURES!
[...truncated 44530 lines...]
[JENKINS] Lucene-Solr-7.x-MacOSX (64bit/jdk1.8.0) - Build # 61 -
Failure!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-MacOSX/61/
Java: 64bit/jdk1.8.0 -XX:-UseCompressedOops -XX:+UseConcMarkSweepGC
1 tests failed.
FAILED: org.apache.solr.cloud.TestPullReplica.testCreateDelete { seed=[B1C77641C05E3C78:1E813BF1026C0F64]}
Error Message:
Error from server at http://127.0.0.1:61139/solr: create the collection time out:180s
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:61139/solr: create the collection time out:180s
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:626)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:252)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:241)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1121)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:862)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:793)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:178)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:195)
at org.apache.solr.cloud.TestPullReplica.testCreateDelete(TestPullReplica.java:129)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 11531 lines...]
[junit4] Suite: org.apache.solr.cloud.TestPullReplica
[junit4] 2> 1270025 INFO (SUITE-TestPullReplica-seed#[B1C77641C05E3C78]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplica_B1C77641C05E3C78-001/init-core-data-001
[junit4] 2> 1270025 WARN (SUITE-TestPullReplica-seed#[B1C77641C05E3C78]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=14 numCloses=14
[junit4] 2> 1270026 INFO (SUITE-TestPullReplica-seed#[B1C77641C05E3C78]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 1270029 INFO (SUITE-TestPullReplica-seed#[B1C77641C05E3C78]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN) w/ MAC_OS_X supressed clientAuth
[junit4] 2> 1270030 INFO (SUITE-TestPullReplica-seed#[B1C77641C05E3C78]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplica_B1C77641C05E3C78-001/tempDir-001
[junit4] 2> 1270030 INFO (SUITE-TestPullReplica-seed#[B1C77641C05E3C78]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1270030 INFO (Thread-1337) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1270030 INFO (Thread-1337) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1270033 ERROR (Thread-1337) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1270136 INFO (SUITE-TestPullReplica-seed#[B1C77641C05E3C78]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:61134
[junit4] 2> 1270240 INFO (jetty-launcher-871-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1270240 INFO (jetty-launcher-871-thread-2) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1270244 INFO (jetty-launcher-871-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@50739880{/solr,null,AVAILABLE}
[junit4] 2> 1270244 INFO (jetty-launcher-871-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6d0320e0{/solr,null,AVAILABLE}
[junit4] 2> 1270245 INFO (jetty-launcher-871-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6eef3993{HTTP/1.1,[http/1.1]}{127.0.0.1:61138}
[junit4] 2> 1270246 INFO (jetty-launcher-871-thread-2) [ ] o.e.j.s.Server Started @1274863ms
[junit4] 2> 1270246 INFO (jetty-launcher-871-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=61138}
[junit4] 2> 1270246 ERROR (jetty-launcher-871-thread-2) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1270246 INFO (jetty-launcher-871-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.1.0
[junit4] 2> 1270247 INFO (jetty-launcher-871-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1270247 INFO (jetty-launcher-871-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 1270247 INFO (jetty-launcher-871-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-07-21T23:53:54.032Z
[junit4] 2> 1270247 INFO (jetty-launcher-871-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@663a2a59{HTTP/1.1,[http/1.1]}{127.0.0.1:61139}
[junit4] 2> 1270247 INFO (jetty-launcher-871-thread-1) [ ] o.e.j.s.Server Started @1274865ms
[junit4] 2> 1270247 INFO (jetty-launcher-871-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=61139}
[junit4] 2> 1270247 ERROR (jetty-launcher-871-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1270247 INFO (jetty-launcher-871-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.1.0
[junit4] 2> 1270247 INFO (jetty-launcher-871-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1270248 INFO (jetty-launcher-871-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 1270248 INFO (jetty-launcher-871-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-07-21T23:53:54.033Z
[junit4] 2> 1270251 INFO (jetty-launcher-871-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1270252 INFO (jetty-launcher-871-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1270270 INFO (jetty-launcher-871-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:61134/solr
[junit4] 2> 1270273 INFO (jetty-launcher-871-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:61134/solr
[junit4] 2> 1270499 INFO (jetty-launcher-871-thread-2) [n:127.0.0.1:61138_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1270499 INFO (jetty-launcher-871-thread-1) [n:127.0.0.1:61139_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1270501 INFO (jetty-launcher-871-thread-2) [n:127.0.0.1:61138_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:61138_solr
[junit4] 2> 1270502 INFO (jetty-launcher-871-thread-1) [n:127.0.0.1:61139_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61139_solr
[junit4] 2> 1270502 INFO (jetty-launcher-871-thread-2) [n:127.0.0.1:61138_solr ] o.a.s.c.Overseer Overseer (id=98348645339430918-127.0.0.1:61138_solr-n_0000000000) starting
[junit4] 2> 1270505 INFO (zkCallback-883-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1270506 INFO (zkCallback-882-thread-1-processing-n:127.0.0.1:61139_solr) [n:127.0.0.1:61139_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1270520 INFO (jetty-launcher-871-thread-2) [n:127.0.0.1:61138_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61138_solr
[junit4] 2> 1270522 INFO (zkCallback-882-thread-1-processing-n:127.0.0.1:61139_solr) [n:127.0.0.1:61139_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1270522 INFO (zkCallback-883-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1270617 INFO (jetty-launcher-871-thread-2) [n:127.0.0.1:61138_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_61138.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@789f2918
[junit4] 2> 1270619 INFO (jetty-launcher-871-thread-1) [n:127.0.0.1:61139_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_61139.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@789f2918
[junit4] 2> 1270621 INFO (jetty-launcher-871-thread-2) [n:127.0.0.1:61138_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_61138.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@789f2918
[junit4] 2> 1270621 INFO (jetty-launcher-871-thread-2) [n:127.0.0.1:61138_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_61138.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@789f2918
[junit4] 2> 1270622 INFO (jetty-launcher-871-thread-1) [n:127.0.0.1:61139_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_61139.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@789f2918
[junit4] 2> 1270622 INFO (jetty-launcher-871-thread-1) [n:127.0.0.1:61139_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_61139.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@789f2918
[junit4] 2> 1270622 INFO (jetty-launcher-871-thread-2) [n:127.0.0.1:61138_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplica_B1C77641C05E3C78-001/tempDir-001/node2/.
[junit4] 2> 1270622 INFO (jetty-launcher-871-thread-1) [n:127.0.0.1:61139_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplica_B1C77641C05E3C78-001/tempDir-001/node1/.
[junit4] 2> 1270655 INFO (SUITE-TestPullReplica-seed#[B1C77641C05E3C78]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 1270657 INFO (SUITE-TestPullReplica-seed#[B1C77641C05E3C78]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:61134/solr ready
[junit4] 2> 1270662 INFO (SUITE-TestPullReplica-seed#[B1C77641C05E3C78]-worker) [ ] o.a.s.c.TestPullReplica Using legacyCloud?: false
[junit4] 2> 1270665 INFO (qtp1721779977-7624) [n:127.0.0.1:61139_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :clusterprop with params val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1270666 INFO (qtp1721779977-7624) [n:127.0.0.1:61139_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 1270679 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testKillLeader
[junit4] 2> 1270681 INFO (qtp1721779977-7625) [n:127.0.0.1:61139_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=1&replicationFactor=1&collection.configName=conf&maxShardsPerNode=100&name=pull_replica_test_kill_leader&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1270683 INFO (OverseerThreadFactory-2826-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr ] o.a.s.c.CreateCollectionCmd Create collection pull_replica_test_kill_leader
[junit4] 2> 1270804 INFO (OverseerStateUpdate-98348645339430918-127.0.0.1:61138_solr-n_0000000000) [n:127.0.0.1:61138_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"pull_replica_test_kill_leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"pull_replica_test_kill_leader_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:61138/solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 1270813 INFO (OverseerStateUpdate-98348645339430918-127.0.0.1:61138_solr-n_0000000000) [n:127.0.0.1:61138_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"pull_replica_test_kill_leader",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"pull_replica_test_kill_leader_shard1_replica_p1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:61139/solr",
[junit4] 2> "type":"PULL"}
[junit4] 2> 1271030 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=pull_replica_test_kill_leader_shard1_replica_p1&action=CREATE&numShards=1&collection=pull_replica_test_kill_leader&shard=shard1&wt=javabin&version=2&replicaType=PULL
[junit4] 2> 1271030 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node1&collection.configName=conf&newCollection=true&name=pull_replica_test_kill_leader_shard1_replica_n1&action=CREATE&numShards=1&collection=pull_replica_test_kill_leader&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1271031 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1271031 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1271160 INFO (zkCallback-883-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1271160 INFO (zkCallback-882-thread-1-processing-n:127.0.0.1:61139_solr) [n:127.0.0.1:61139_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1271160 INFO (zkCallback-883-thread-2-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1271161 INFO (zkCallback-882-thread-2-processing-n:127.0.0.1:61139_solr) [n:127.0.0.1:61139_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1272072 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 1272073 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 1272082 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.s.IndexSchema [pull_replica_test_kill_leader_shard1_replica_p1] Schema name=minimal
[junit4] 2> 1272083 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.s.IndexSchema [pull_replica_test_kill_leader_shard1_replica_n1] Schema name=minimal
[junit4] 2> 1272091 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1272091 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1272091 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.CoreContainer Creating SolrCore 'pull_replica_test_kill_leader_shard1_replica_p1' using configuration from collection pull_replica_test_kill_leader, trusted=true
[junit4] 2> 1272091 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'pull_replica_test_kill_leader_shard1_replica_n1' using configuration from collection pull_replica_test_kill_leader, trusted=true
[junit4] 2> 1272092 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_61138.solr.core.pull_replica_test_kill_leader.shard1.replica_n1' (registry 'solr.core.pull_replica_test_kill_leader.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@789f2918
[junit4] 2> 1272092 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_61139.solr.core.pull_replica_test_kill_leader.shard1.replica_p1' (registry 'solr.core.pull_replica_test_kill_leader.shard1.replica_p1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@789f2918
[junit4] 2> 1272092 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1272092 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.SolrCore [[pull_replica_test_kill_leader_shard1_replica_n1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplica_B1C77641C05E3C78-001/tempDir-001/node2/pull_replica_test_kill_leader_shard1_replica_n1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplica_B1C77641C05E3C78-001/tempDir-001/node2/./pull_replica_test_kill_leader_shard1_replica_n1/data/]
[junit4] 2> 1272092 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1272092 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.SolrCore [[pull_replica_test_kill_leader_shard1_replica_p1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplica_B1C77641C05E3C78-001/tempDir-001/node1/pull_replica_test_kill_leader_shard1_replica_p1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplica_B1C77641C05E3C78-001/tempDir-001/node1/./pull_replica_test_kill_leader_shard1_replica_p1/data/]
[junit4] 2> 1272149 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1272149 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1272149 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1272149 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1272150 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.s.SolrIndexSearcher Opening [Searcher@c3056bf[pull_replica_test_kill_leader_shard1_replica_p1] main]
[junit4] 2> 1272150 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1272150 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1272151 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1272151 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@ff33d92[pull_replica_test_kill_leader_shard1_replica_n1] main]
[junit4] 2> 1272151 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1272152 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1272153 INFO (searcherExecutor-2831-thread-1-processing-n:127.0.0.1:61139_solr x:pull_replica_test_kill_leader_shard1_replica_p1 s:shard1 c:pull_replica_test_kill_leader r:core_node2) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.SolrCore [pull_replica_test_kill_leader_shard1_replica_p1] Registered new searcher Searcher@c3056bf[pull_replica_test_kill_leader_shard1_replica_p1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1272153 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1272153 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.ZkController pull_replica_test_kill_leader_shard1_replica_p1 starting background replication from leader
[junit4] 2> 1272153 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
[junit4] 2> 1272153 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1272154 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1272154 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1573578327655972864
[junit4] 2> 1272155 INFO (searcherExecutor-2832-thread-1-processing-n:127.0.0.1:61138_solr x:pull_replica_test_kill_leader_shard1_replica_n1 s:shard1 c:pull_replica_test_kill_leader r:core_node1) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.SolrCore [pull_replica_test_kill_leader_shard1_replica_n1] Registered new searcher Searcher@ff33d92[pull_replica_test_kill_leader_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1272156 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
[junit4] 2> 1272161 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1272161 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1272161 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:61138/solr/pull_replica_test_kill_leader_shard1_replica_n1/
[junit4] 2> 1272161 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1272161 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:61138/solr/pull_replica_test_kill_leader_shard1_replica_n1/ has no replicas
[junit4] 2> 1272161 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1272166 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:61138/solr/pull_replica_test_kill_leader_shard1_replica_n1/ shard1
[junit4] 2> 1272278 INFO (zkCallback-882-thread-2-processing-n:127.0.0.1:61139_solr) [n:127.0.0.1:61139_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1272278 INFO (zkCallback-882-thread-1-processing-n:127.0.0.1:61139_solr) [n:127.0.0.1:61139_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1272278 INFO (zkCallback-883-thread-2-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1272278 INFO (zkCallback-883-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1272335 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1272337 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node1&collection.configName=conf&newCollection=true&name=pull_replica_test_kill_leader_shard1_replica_n1&action=CREATE&numShards=1&collection=pull_replica_test_kill_leader&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1307
[junit4] 2> 1272451 INFO (zkCallback-882-thread-1-processing-n:127.0.0.1:61139_solr) [n:127.0.0.1:61139_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1272451 INFO (zkCallback-883-thread-2-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1272451 INFO (zkCallback-882-thread-2-processing-n:127.0.0.1:61139_solr) [n:127.0.0.1:61139_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1272451 INFO (zkCallback-883-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1273170 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf&newCollection=true&name=pull_replica_test_kill_leader_shard1_replica_p1&action=CREATE&numShards=1&collection=pull_replica_test_kill_leader&shard=shard1&wt=javabin&version=2&replicaType=PULL} status=0 QTime=2140
[junit4] 2> 1273177 INFO (qtp1721779977-7625) [n:127.0.0.1:61139_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1273280 INFO (zkCallback-882-thread-1-processing-n:127.0.0.1:61139_solr) [n:127.0.0.1:61139_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1273280 INFO (zkCallback-882-thread-2-processing-n:127.0.0.1:61139_solr) [n:127.0.0.1:61139_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1273280 INFO (zkCallback-883-thread-2-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1273280 INFO (zkCallback-883-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1274169 INFO (indexFetcher-2841-thread-1) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.h.IndexFetcher Updated masterUrl to http://127.0.0.1:61138/solr/pull_replica_test_kill_leader_shard1_replica_n1/
[junit4] 2> 1274170 INFO (qtp2055047171-7634) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_n1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 1274170 INFO (indexFetcher-2841-thread-1) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 1274170 INFO (indexFetcher-2841-thread-1) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 1274170 INFO (indexFetcher-2841-thread-1) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 1274170 INFO (indexFetcher-2841-thread-1) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 1274186 INFO (qtp1721779977-7625) [n:127.0.0.1:61139_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=1&replicationFactor=1&collection.configName=conf&maxShardsPerNode=100&name=pull_replica_test_kill_leader&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2} status=0 QTime=3505
[junit4] 2> 1274197 INFO (qtp2055047171-7636) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [pull_replica_test_kill_leader_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1 (1573578329795067904)]} 0 3
[junit4] 2> 1274200 INFO (qtp2055047171-7637) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1573578329801359360,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1274200 INFO (qtp2055047171-7637) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@4e9eef1e commitCommandVersion:1573578329801359360
[junit4] 2> 1274206 INFO (qtp2055047171-7637) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@322dab58[pull_replica_test_kill_leader_shard1_replica_n1] main]
[junit4] 2> 1274206 INFO (qtp2055047171-7637) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1274207 INFO (searcherExecutor-2832-thread-1-processing-n:127.0.0.1:61138_solr x:pull_replica_test_kill_leader_shard1_replica_n1 s:shard1 c:pull_replica_test_kill_leader r:core_node1) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.SolrCore [pull_replica_test_kill_leader_shard1_replica_n1] Registered new searcher Searcher@322dab58[pull_replica_test_kill_leader_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.1.0):C1)))}
[junit4] 2> 1274207 INFO (qtp2055047171-7637) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [pull_replica_test_kill_leader_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=pull_replica_test_kill_leader:5&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 7
[junit4] 2> 1274210 INFO (qtp2055047171-7630) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=0
[junit4] 2> 1274211 INFO (qtp1721779977-7628) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1274322 INFO (qtp1721779977-7622) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1274434 INFO (qtp1721779977-7624) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1274546 INFO (qtp1721779977-7626) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1274657 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1274708 INFO (OverseerCollectionConfigSetProcessor-98348645339430918-127.0.0.1:61138_solr-n_0000000000) [n:127.0.0.1:61138_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1274764 INFO (qtp1721779977-7625) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1274876 INFO (qtp1721779977-7629) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1274987 INFO (qtp1721779977-7628) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1275097 INFO (qtp1721779977-7622) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1275206 INFO (qtp1721779977-7624) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1275321 INFO (qtp1721779977-7626) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1275432 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1275542 INFO (qtp1721779977-7625) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1275654 INFO (qtp1721779977-7629) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1275765 INFO (qtp1721779977-7628) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1275876 INFO (qtp1721779977-7622) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1275987 INFO (qtp1721779977-7624) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1276099 INFO (qtp1721779977-7626) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1276210 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1276320 INFO (qtp1721779977-7625) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1276432 INFO (qtp1721779977-7629) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1276543 INFO (qtp1721779977-7628) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1276655 INFO (qtp1721779977-7622) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1276761 INFO (qtp1721779977-7624) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1276869 INFO (qtp1721779977-7626) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1276973 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1277080 INFO (qtp1721779977-7625) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1277172 INFO (qtp2055047171-7633) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_n1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 1277172 INFO (indexFetcher-2841-thread-1) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.h.IndexFetcher Master's generation: 2
[junit4] 2> 1277172 INFO (indexFetcher-2841-thread-1) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.h.IndexFetcher Master's version: 1500681237985
[junit4] 2> 1277172 INFO (indexFetcher-2841-thread-1) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 1277172 INFO (indexFetcher-2841-thread-1) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 1277173 INFO (indexFetcher-2841-thread-1) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.h.IndexFetcher Starting replication process
[junit4] 2> 1277174 INFO (qtp2055047171-7635) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_n1] webapp=/solr path=/replication params={generation=2&qt=/replication&wt=javabin&version=2&command=filelist} status=0 QTime=0
[junit4] 2> 1277174 INFO (indexFetcher-2841-thread-1) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.h.IndexFetcher Number of files in latest index in master: 15
[junit4] 2> 1277176 INFO (indexFetcher-2841-thread-1) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MockDirectoryWrapper(RAMDirectory@38fb90be lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2292b5c3)
[junit4] 2> 1277176 INFO (qtp2055047171-7636) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_n1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 1277178 INFO (qtp2055047171-7637) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_n1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 1277180 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_n1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 1277181 INFO (qtp2055047171-7630) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_n1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 1277182 INFO (qtp2055047171-7630) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_n1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 1277182 INFO (qtp1721779977-7629) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 1277183 INFO (qtp2055047171-7633) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_n1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 1277184 INFO (qtp2055047171-7635) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_n1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 1277185 INFO (qtp2055047171-7636) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_n1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 1277185 INFO (qtp2055047171-7637) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_n1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene70_0.dvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 1277186 INFO (qtp2055047171-7632) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_n1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.dii&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 1277186 INFO (qtp2055047171-7634) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_n1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Lucene70_0.dvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 1277187 INFO (qtp2055047171-7630) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_n1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.dim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 1277188 INFO (qtp2055047171-7633) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_n1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 1277188 INFO (qtp2055047171-7635) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_n1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fnm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 1277189 INFO (qtp2055047171-7636) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_n1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=segments_2&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 1277189 INFO (indexFetcher-2841-thread-1) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.h.IndexFetcher Total time taken for download (fullCopy=false,bytesDownloaded=2118) : 0 secs (null bytes/sec) to MockDirectoryWrapper(RAMDirectory@38fb90be lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2292b5c3)
[junit4] 2> 1277191 INFO (indexFetcher-2841-thread-1) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 1277192 INFO (indexFetcher-2841-thread-1) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6763e78b[pull_replica_test_kill_leader_shard1_replica_p1] main]
[junit4] 2> 1277193 INFO (searcherExecutor-2831-thread-1-processing-n:127.0.0.1:61139_solr x:pull_replica_test_kill_leader_shard1_replica_p1 s:shard1 c:pull_replica_test_kill_leader r:core_node2) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.SolrCore [pull_replica_test_kill_leader_shard1_replica_p1] Registered new searcher Searcher@6763e78b[pull_replica_test_kill_leader_shard1_replica_p1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.1.0):C1)))}
[junit4] 2> 1277290 INFO (qtp1721779977-7628) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=0
[junit4] 2> 1277291 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.a.s.c.ChaosMonkey monkey: kill jetty! 61138
[junit4] 2> 1277293 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@6eef3993{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 1277295 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=52550251
[junit4] 2> 1277295 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1277299 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1277302 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1277306 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1277308 INFO (coreCloseExecutor-2843-thread-1) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.SolrCore [pull_replica_test_kill_leader_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@237a8949
[junit4] 2> 1277312 INFO (coreCloseExecutor-2843-thread-1) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.pull_replica_test_kill_leader.shard1.replica_n1, tag=595233097
[junit4] 2> 1277316 INFO (zkCallback-882-thread-2-processing-n:127.0.0.1:61139_solr) [n:127.0.0.1:61139_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1277316 INFO (zkCallback-882-thread-1-processing-n:127.0.0.1:61139_solr) [n:127.0.0.1:61139_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1277316 INFO (zkCallback-883-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1277316 INFO (zkCallback-883-thread-2-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1277316 INFO (zkCallback-889-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1277318 INFO (coreCloseExecutor-2843-thread-1) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.pull_replica_test_kill_leader.shard1.leader, tag=595233097
[junit4] 2> 1277320 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.a.s.c.Overseer Overseer (id=98348645339430918-127.0.0.1:61138_solr-n_0000000000) closing
[junit4] 2> 1277321 INFO (OverseerStateUpdate-98348645339430918-127.0.0.1:61138_solr-n_0000000000) [n:127.0.0.1:61138_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:61138_solr
[junit4] 2> 1277324 WARN (zkCallback-883-thread-2-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 1277325 INFO (zkCallback-889-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 1277325 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@50739880{/solr,null,UNAVAILABLE}
[junit4] 2> 1277325 INFO (zkCallback-882-thread-2-processing-n:127.0.0.1:61139_solr) [n:127.0.0.1:61139_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
[junit4] 2> 1277326 INFO (zkCallback-882-thread-1-processing-n:127.0.0.1:61139_solr) [n:127.0.0.1:61139_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:61139_solr
[junit4] 2> 1277327 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.a.s.c.ChaosMonkey monkey: stop jetty! 61138
[junit4] 2> 1277329 INFO (zkCallback-882-thread-1-processing-n:127.0.0.1:61139_solr) [n:127.0.0.1:61139_solr ] o.a.s.c.Overseer Overseer (id=98348645339430917-127.0.0.1:61139_solr-n_0000000001) starting
[junit4] 2> 1281353 INFO (qtp1721779977-7622) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.u.p.LogUpdateProcessorFactory [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/update params={wt=javabin&version=2}{} 0 4019
[junit4] 2> 1281354 INFO (qtp1721779977-7622) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.h.RequestHandlerBase Ignoring exception matching No registered leader was found
[junit4] 2> 1281355 ERROR (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.a.s.c.s.i.CloudSolrClient Request to collection pull_replica_test_kill_leader failed due to (503) org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:61139/solr/pull_replica_test_kill_leader_shard1_replica_p1: No registered leader was found after waiting for 4000ms , collection: pull_replica_test_kill_leader slice: shard1, retry? 0
[junit4] 2> 1284208 INFO (indexFetcher-2841-thread-1) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.h.ReplicationHandler Ignoring exception matching No registered leader was found
[junit4] 2> 1284208 INFO (indexFetcher-2841-thread-1) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.h.IndexFetcher Last replication failed, so I'll force replication
[junit4] 2> 1285364 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.u.p.LogUpdateProcessorFactory [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/update params={wt=javabin&version=2}{} 0 4007
[junit4] 2> 1285365 INFO (qtp1721779977-7627) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.h.RequestHandlerBase Ignoring exception matching No registered leader was found
[junit4] 2> 1285365 ERROR (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.a.s.c.s.i.CloudSolrClient Request to collection pull_replica_test_kill_leader failed due to (503) org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:61139/solr/pull_replica_test_kill_leader_shard1_replica_p1: No registered leader was found after waiting for 4000ms , collection: pull_replica_test_kill_leader slice: shard1, retry? 0
[junit4] 2> 1285367 INFO (qtp1721779977-7625) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.c.S.Request [pull_replica_test_kill_leader_shard1_replica_p1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=0
[junit4] 2> 1285367 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.a.s.c.ChaosMonkey monkey: starting jetty! 61138
[junit4] 2> 1285368 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1285369 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@678c345{/solr,null,AVAILABLE}
[junit4] 2> 1285369 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7477c43c{HTTP/1.1,[http/1.1]}{127.0.0.1:61138}
[junit4] 2> 1285369 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.e.j.s.Server Started @1289987ms
[junit4] 2> 1285369 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=61138}
[junit4] 2> 1285369 ERROR (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1285370 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.1.0
[junit4] 2> 1285370 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1285370 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 1285370 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-07-21T23:54:09.155Z
[junit4] 2> 1285371 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1285382 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:61134/solr
[junit4] 2> 1285393 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [n:127.0.0.1:61138_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1285396 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [n:127.0.0.1:61138_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1285402 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [n:127.0.0.1:61138_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:61138_solr
[junit4] 2> 1285403 INFO (zkCallback-889-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1285404 INFO (zkCallback-882-thread-1-processing-n:127.0.0.1:61139_solr) [n:127.0.0.1:61139_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1285403 INFO (zkCallback-895-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 1285405 INFO (zkCallback-882-thread-1-processing-n:127.0.0.1:61139_solr) [n:127.0.0.1:61139_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1285405 INFO (zkCallback-882-thread-2-processing-n:127.0.0.1:61139_solr) [n:127.0.0.1:61139_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1285405 INFO (zkCallback-889-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1285416 INFO (indexFetcher-2841-thread-1) [n:127.0.0.1:61139_solr c:pull_replica_test_kill_leader s:shard1 r:core_node2 x:pull_replica_test_kill_leader_shard1_replica_p1] o.a.s.h.IndexFetcher Replica core_node1 is leader but it's state is down, skipping replication
[junit4] 2> 1285510 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [n:127.0.0.1:61138_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_61138.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@789f2918
[junit4] 2> 1285513 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [n:127.0.0.1:61138_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_61138.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@789f2918
[junit4] 2> 1285513 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [n:127.0.0.1:61138_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_61138.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@789f2918
[junit4] 2> 1285515 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [n:127.0.0.1:61138_solr ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplica_B1C77641C05E3C78-001/tempDir-001/node2/.
[junit4] 2> 1285515 INFO (TEST-TestPullReplica.testKillLeader-seed#[B1C77641C05E3C78]) [n:127.0.0.1:61138_solr ] o.a.s.c.CorePropertiesLocator Cores are: [pull_replica_test_kill_leader_shard1_replica_n1]
[junit4] 2> 1285517 INFO (coreLoadExecutor-2852-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1285539 INFO (coreLoadExecutor-2852-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.1.0
[junit4] 2> 1285546 INFO (coreLoadExecutor-2852-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.s.IndexSchema [pull_replica_test_kill_leader_shard1_replica_n1] Schema name=minimal
[junit4] 2> 1285550 INFO (coreLoadExecutor-2852-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1285550 INFO (coreLoadExecutor-2852-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'pull_replica_test_kill_leader_shard1_replica_n1' using configuration from collection pull_replica_test_kill_leader, trusted=true
[junit4] 2> 1285551 INFO (coreLoadExecutor-2852-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_61138.solr.core.pull_replica_test_kill_leader.shard1.replica_n1' (registry 'solr.core.pull_replica_test_kill_leader.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@789f2918
[junit4] 2> 1285551 INFO (coreLoadExecutor-2852-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1285551 INFO (coreLoadExecutor-2852-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.SolrCore [[pull_replica_test_kill_leader_shard1_replica_n1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplica_B1C77641C05E3C78-001/tempDir-001/node2/pull_replica_test_kill_leader_shard1_replica_n1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestPullReplica_B1C77641C05E3C78-001/tempDir-001/node2/./pull_replica_test_kill_leader_shard1_replica_n1/data/]
[junit4] 2> 1285593 INFO (coreLoadExecutor-2852-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1285593 INFO (coreLoadExecutor-2852-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1285594 INFO (coreLoadExecutor-2852-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1285594 INFO (coreLoadExecutor-2852-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1285595 INFO (coreLoadExecutor-2852-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@8a56d71[pull_replica_test_kill_leader_shard1_replica_n1] main]
[junit4] 2> 1285596 INFO (coreLoadExecutor-2852-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 1285596 INFO (coreLoadExecutor-2852-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 1285597 INFO (coreLoadExecutor-2852-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1285597 INFO (coreLoadExecutor-2852-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1573578341751980032
[junit4] 2> 1285598 INFO (searcherExecutor-2853-thread-1-processing-n:127.0.0.1:61138_solr x:pull_replica_test_kill_leader_shard1_replica_n1 s:shard1 c:pull_replica_test_kill_leader r:core_node1) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.SolrCore [pull_replica_test_kill_leader_shard1_replica_n1] Registered new searcher Searcher@8a56d71[pull_replica_test_kill_leader_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1285601 INFO (coreZkRegister-2847-thread-1-processing-n:127.0.0.1:61138_solr x:pull_replica_test_kill_leader_shard1_replica_n1 s:shard1 c:pull_replica_test_kill_leader r:core_node1) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1285602 INFO (coreZkRegister-2847-thread-1-processing-n:127.0.0.1:61138_solr x:pull_replica_test_kill_leader_shard1_replica_n1 s:shard1 c:pull_replica_test_kill_leader r:core_node1) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1285602 INFO (coreZkRegister-2847-thread-1-processing-n:127.0.0.1:61138_solr x:pull_replica_test_kill_leader_shard1_replica_n1 s:shard1 c:pull_replica_test_kill_leader r:core_node1) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:61138/solr/pull_replica_test_kill_leader_shard1_replica_n1/
[junit4] 2> 1285602 INFO (coreZkRegister-2847-thread-1-processing-n:127.0.0.1:61138_solr x:pull_replica_test_kill_leader_shard1_replica_n1 s:shard1 c:pull_replica_test_kill_leader r:core_node1) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1285602 INFO (coreZkRegister-2847-thread-1-processing-n:127.0.0.1:61138_solr x:pull_replica_test_kill_leader_shard1_replica_n1 s:shard1 c:pull_replica_test_kill_leader r:core_node1) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:61138/solr/pull_replica_test_kill_leader_shard1_replica_n1/ has no replicas
[junit4] 2> 1285602 INFO (coreZkRegister-2847-thread-1-processing-n:127.0.0.1:61138_solr x:pull_replica_test_kill_leader_shard1_replica_n1 s:shard1 c:pull_replica_test_kill_leader r:core_node1) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1285604 INFO (coreZkRegister-2847-thread-1-processing-n:127.0.0.1:61138_solr x:pull_replica_test_kill_leader_shard1_replica_n1 s:shard1 c:pull_replica_test_kill_leader r:core_node1) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:61138/solr/pull_replica_test_kill_leader_shard1_replica_n1/ shard1
[junit4] 2> 1285605 INFO (coreZkRegister-2847-thread-1-processing-n:127.0.0.1:61138_solr x:pull_replica_test_kill_leader_shard1_replica_n1 s:shard1 c:pull_replica_test_kill_leader r:core_node1) [n:127.0.0.1:61138_solr c:pull_replica_test_kill_leader s:shard1 r:core_node1 x:pull_replica_test_kill_leader_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1285721 INFO (zkCallback-882-thread-2-processing-n:127.0.0.1:61139_solr) [n:127.0.0.1:61139_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1285721 INFO (zkCallback-889-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1285721 INFO (zkCallback-882-thread-1-processing-n:127.0.0.1:61139_solr) [n:127.0.0.1:61139_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leader/state.json] for collection [pull_replica_test_kill_leader] has occurred - updating... (live nodes size: [2])
[junit4] 2> 1285723 INFO (zkCallback-895-thread-1-processing-n:127.0.0.1:61138_solr) [n:127.0.0.1:61138_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/pull_replica_test_kill_leade
[...truncated too long message...]
eak: 'client' is never closed
[ecj-lint] ----------
[ecj-lint] 10. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/handler/component/DistributedDebugComponentTest.java (at line 221)
[ecj-lint] throw new AssertionError(q.toString() + ": " + e.getMessage(), e);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'client' is not closed at this location
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 11. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/highlight/HighlighterTest.java (at line 204)
[ecj-lint] Analyzer a1 = new WhitespaceAnalyzer();
[ecj-lint] ^^
[ecj-lint] Resource leak: 'a1' is never closed
[ecj-lint] ----------
[ecj-lint] 12. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/highlight/HighlighterTest.java (at line 207)
[ecj-lint] OffsetWindowTokenFilter tots = new OffsetWindowTokenFilter(tokenStream);
[ecj-lint] ^^^^
[ecj-lint] Resource leak: 'tots' is never closed
[ecj-lint] ----------
[ecj-lint] 13. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/highlight/HighlighterTest.java (at line 211)
[ecj-lint] Analyzer a2 = new WhitespaceAnalyzer();
[ecj-lint] ^^
[ecj-lint] Resource leak: 'a2' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 14. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/response/TestJavabinTupleStreamParser.java (at line 72)
[ecj-lint] JavabinTupleStreamParser parser = new JavabinTupleStreamParser(new ByteArrayInputStream(bytes), true);
[ecj-lint] ^^^^^^
[ecj-lint] Resource leak: 'parser' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 15. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/search/TestDocSet.java (at line 243)
[ecj-lint] return loadfactor!=0 ? new HashDocSet(a,0,n,1/loadfactor) : new HashDocSet(a,0,n);
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] 16. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/search/TestDocSet.java (at line 528)
[ecj-lint] DocSet a = new BitDocSet(bs);
[ecj-lint] ^
[ecj-lint] Resource leak: 'a' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 17. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/security/TestPKIAuthenticationPlugin.java (at line 76)
[ecj-lint] final MockPKIAuthenticationPlugin mock = new MockPKIAuthenticationPlugin(null, nodeName);
[ecj-lint] ^^^^
[ecj-lint] Resource leak: 'mock' is never closed
[ecj-lint] ----------
[ecj-lint] 18. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/security/TestPKIAuthenticationPlugin.java (at line 131)
[ecj-lint] MockPKIAuthenticationPlugin mock1 = new MockPKIAuthenticationPlugin(null, nodeName) {
[ecj-lint] ^^^^^
[ecj-lint] Resource leak: 'mock1' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 19. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/security/TestRuleBasedAuthorizationPlugin.java (at line 380)
[ecj-lint] RuleBasedAuthorizationPlugin plugin = new RuleBasedAuthorizationPlugin();
[ecj-lint] ^^^^^^
[ecj-lint] Resource leak: 'plugin' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 20. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/security/TestSha256AuthenticationProvider.java (at line 49)
[ecj-lint] BasicAuthPlugin basicAuthPlugin = new BasicAuthPlugin();
[ecj-lint] ^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: 'basicAuthPlugin' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 21. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/spelling/SimpleQueryConverter.java (at line 42)
[ecj-lint] WhitespaceAnalyzer analyzer = new WhitespaceAnalyzer();
[ecj-lint] ^^^^^^^^
[ecj-lint] Resource leak: 'analyzer' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 22. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/uninverting/TestFieldCacheVsDocValues.java (at line 139)
[ecj-lint] IndexWriter w = new IndexWriter(d, newIndexWriterConfig(analyzer));
[ecj-lint] ^
[ecj-lint] Resource leak: 'w' is never closed
[ecj-lint] ----------
[ecj-lint] 23. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/uninverting/TestFieldCacheVsDocValues.java (at line 172)
[ecj-lint] throw iae;
[ecj-lint] ^^^^^^^^^^
[ecj-lint] Resource leak: 'w' is not closed at this location
[ecj-lint] ----------
[ecj-lint] 24. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/uninverting/TestFieldCacheVsDocValues.java (at line 178)
[ecj-lint] return;
[ecj-lint] ^^^^^^^
[ecj-lint] Resource leak: 'w' is not closed at this location
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 25. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/update/SolrCmdDistributorTest.java (at line 134)
[ecj-lint] SolrCmdDistributor cmdDistrib = new SolrCmdDistributor(updateShardHandler);
[ecj-lint] ^^^^^^^^^^
[ecj-lint] Resource leak: 'cmdDistrib' is never closed
[ecj-lint] ----------
[ecj-lint] 26. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/update/SolrCmdDistributorTest.java (at line 333)
[ecj-lint] SolrCmdDistributor cmdDistrib = new SolrCmdDistributor(streamingClients, 5, 0);
[ecj-lint] ^^^^^^^^^^
[ecj-lint] Resource leak: 'cmdDistrib' is never closed
[ecj-lint] ----------
[ecj-lint] 27. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/update/SolrCmdDistributorTest.java (at line 367)
[ecj-lint] SolrCmdDistributor cmdDistrib = new SolrCmdDistributor(streamingClients, 5, 0);
[ecj-lint] ^^^^^^^^^^
[ecj-lint] Resource leak: 'cmdDistrib' is never closed
[ecj-lint] ----------
[ecj-lint] 28. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/update/SolrCmdDistributorTest.java (at line 413)
[ecj-lint] SolrCmdDistributor cmdDistrib = new SolrCmdDistributor(streamingClients, 5, 0);
[ecj-lint] ^^^^^^^^^^
[ecj-lint] Resource leak: 'cmdDistrib' is never closed
[ecj-lint] ----------
[ecj-lint] 29. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/update/SolrCmdDistributorTest.java (at line 458)
[ecj-lint] SolrCmdDistributor cmdDistrib = new SolrCmdDistributor(updateShardHandler);
[ecj-lint] ^^^^^^^^^^
[ecj-lint] Resource leak: 'cmdDistrib' is never closed
[ecj-lint] ----------
[ecj-lint] 30. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/update/SolrCmdDistributorTest.java (at line 516)
[ecj-lint] SolrCmdDistributor cmdDistrib = new SolrCmdDistributor(updateShardHandler);
[ecj-lint] ^^^^^^^^^^
[ecj-lint] Resource leak: 'cmdDistrib' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 31. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/update/SolrIndexSplitterTest.java (at line 181)
[ecj-lint] EmbeddedSolrServer server1 = new EmbeddedSolrServer(h.getCoreContainer(), "split1");
[ecj-lint] ^^^^^^^
[ecj-lint] Resource leak: 'server1' is never closed
[ecj-lint] ----------
[ecj-lint] 32. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/update/SolrIndexSplitterTest.java (at line 182)
[ecj-lint] EmbeddedSolrServer server2 = new EmbeddedSolrServer(h.getCoreContainer(), "split2");
[ecj-lint] ^^^^^^^
[ecj-lint] Resource leak: 'server2' is never closed
[ecj-lint] ----------
[ecj-lint] ----------
[ecj-lint] 33. WARNING in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/core/src/test/org/apache/solr/update/processor/RecordingUpdateProcessorFactory.java (at line 67)
[ecj-lint] return recording ? new RecordingUpdateRequestProcessor(commandQueue, next) : next;
[ecj-lint] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[ecj-lint] Resource leak: '<unassigned Closeable value>' is never closed
[ecj-lint] ----------
[ecj-lint] 33 problems (3 errors, 30 warnings)
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/build.xml:810: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/build.xml:101: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build.xml:689: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/common-build.xml:2013: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/common-build.xml:2046: Compile failed; see the compiler error output for details.
Total time: 92 minutes 10 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any