You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2017/11/09 21:35:11 UTC
[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk1.8.0_144) - Build # 782
- Still Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/782/
Java: 64bit/jdk1.8.0_144 -XX:-UseCompressedOops -XX:+UseG1GC
1 tests failed.
FAILED: org.apache.solr.cloud.HttpPartitionTest.test
Error Message:
max version bucket seed not updated after recovery!
Stack Trace:
java.lang.AssertionError: max version bucket seed not updated after recovery!
at __randomizedtesting.SeedInfo.seed([DC06EE4BF292881F:5452D1915C6EE5E7]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:377)
at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:132)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13228 lines...]
[junit4] Suite: org.apache.solr.cloud.HttpPartitionTest
[junit4] 2> 2127415 INFO (SUITE-HttpPartitionTest-seed#[DC06EE4BF292881F]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/init-core-data-001
[junit4] 2> 2127415 INFO (SUITE-HttpPartitionTest-seed#[DC06EE4BF292881F]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 2127416 INFO (SUITE-HttpPartitionTest-seed#[DC06EE4BF292881F]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 2127416 INFO (SUITE-HttpPartitionTest-seed#[DC06EE4BF292881F]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /muuv/me
[junit4] 2> 2127418 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2127422 INFO (Thread-3881) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2127422 INFO (Thread-3881) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2127423 ERROR (Thread-3881) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2127518 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.ZkTestServer start zk server on port:40351
[junit4] 2> 2127523 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 2127524 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 2127524 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2127525 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 2127525 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 2127526 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 2127526 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 2127526 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 2127527 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2127527 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 2127527 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 2127528 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use TLOG replicas unless explicitly asked otherwise
[junit4] 2> 2127597 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2127598 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@41984be1{/muuv/me,null,AVAILABLE}
[junit4] 2> 2127600 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@360ea46a{HTTP/1.1,[http/1.1]}{127.0.0.1:46587}
[junit4] 2> 2127600 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.e.j.s.Server Started @2130133ms
[junit4] 2> 2127600 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/tempDir-001/control/data, replicaType=NRT, hostContext=/muuv/me, hostPort=42873, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/control-001/cores}
[junit4] 2> 2127600 ERROR (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2127600 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.2.0
[junit4] 2> 2127601 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2127601 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2127601 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-09T20:56:38.144Z
[junit4] 2> 2127602 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2127602 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/control-001/solr.xml
[junit4] 2> 2127605 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2127605 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2127606 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@501633a1, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2127609 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40351/solr
[junit4] 2> 2127634 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2127634 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:42873_muuv%2Fme
[junit4] 2> 2127634 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.Overseer Overseer (id=98976464762699780-127.0.0.1:42873_muuv%2Fme-n_0000000000) starting
[junit4] 2> 2127639 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:42873_muuv%2Fme
[junit4] 2> 2127647 INFO (zkCallback-3783-thread-1-processing-n:127.0.0.1:42873_muuv%2Fme) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2127687 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@501633a1
[junit4] 2> 2127697 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@501633a1
[junit4] 2> 2127697 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@501633a1
[junit4] 2> 2127699 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/control-001/cores
[junit4] 2> 2127719 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2127719 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40351/solr ready
[junit4] 2> 2127719 INFO (SocketProxy-Acceptor-42873) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=42872,localport=42873], receiveBufferSize:531000
[junit4] 2> 2127720 INFO (SocketProxy-Acceptor-42873) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=46587,localport=46486], receiveBufferSize=530904
[junit4] 2> 2127720 INFO (qtp494783358-18299) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:42873_muuv%252Fme&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2127723 INFO (OverseerThreadFactory-7327-thread-1-processing-n:127.0.0.1:42873_muuv%2Fme) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 2127831 INFO (SocketProxy-Acceptor-42873) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=42876,localport=42873], receiveBufferSize:531000
[junit4] 2> 2127835 INFO (SocketProxy-Acceptor-42873) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=46587,localport=46490], receiveBufferSize=530904
[junit4] 2> 2127836 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2127836 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2127943 INFO (zkCallback-3783-thread-1-processing-n:127.0.0.1:42873_muuv%2Fme) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2128844 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.2.0
[junit4] 2> 2128852 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 2128940 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2128945 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 2128945 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@501633a1
[junit4] 2> 2128945 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2128945 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 2128947 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=16, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 2128948 WARN (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2128968 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2128968 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2128970 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2128970 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2128970 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=98.857421875, floorSegmentMB=1.287109375, forceMergeDeletesPctAllowed=5.777064652578674, segmentsPerTier=36.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5614771407648974
[junit4] 2> 2128970 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3ae56ac5[control_collection_shard1_replica_n1] main]
[junit4] 2> 2128971 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2128971 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2128972 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2128972 INFO (searcherExecutor-7330-thread-1-processing-n:127.0.0.1:42873_muuv%2Fme x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@3ae56ac5[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2128973 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1583623437828489216
[junit4] 2> 2128975 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2128975 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2128975 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:42873/muuv/me/control_collection_shard1_replica_n1/
[junit4] 2> 2128975 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2128975 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:42873/muuv/me/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 2128975 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2128976 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:42873/muuv/me/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 2129077 INFO (zkCallback-3783-thread-2-processing-n:127.0.0.1:42873_muuv%2Fme) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2129127 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2129128 INFO (qtp494783358-18297) [n:127.0.0.1:42873_muuv%2Fme c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1292
[junit4] 2> 2129129 INFO (qtp494783358-18299) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2129228 INFO (zkCallback-3783-thread-2-processing-n:127.0.0.1:42873_muuv%2Fme) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2129724 INFO (OverseerCollectionConfigSetProcessor-98976464762699780-127.0.0.1:42873_muuv%2Fme-n_0000000000) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2130129 INFO (qtp494783358-18299) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:42873_muuv%252Fme&wt=javabin&version=2} status=0 QTime=2409
[junit4] 2> 2130132 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2130133 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40351/solr ready
[junit4] 2> 2130133 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 2130133 INFO (SocketProxy-Acceptor-42873) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=42900,localport=42873], receiveBufferSize:531000
[junit4] 2> 2130133 INFO (SocketProxy-Acceptor-42873) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=46587,localport=46514], receiveBufferSize=530904
[junit4] 2> 2130133 INFO (qtp494783358-18298) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2130134 INFO (OverseerThreadFactory-7327-thread-2-processing-n:127.0.0.1:42873_muuv%2Fme) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 2130134 WARN (OverseerThreadFactory-7327-thread-2-processing-n:127.0.0.1:42873_muuv%2Fme) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 2130336 INFO (qtp494783358-18298) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2130336 INFO (qtp494783358-18298) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=0 QTime=202
[junit4] 2> 2130395 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 1 in directory /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/shard-1-001 of type TLOG
[junit4] 2> 2130395 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2130396 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@628a3a85{/muuv/me,null,AVAILABLE}
[junit4] 2> 2130399 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@28c2223{HTTP/1.1,[http/1.1]}{127.0.0.1:39617}
[junit4] 2> 2130399 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.e.j.s.Server Started @2132932ms
[junit4] 2> 2130399 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/tempDir-001/jetty1, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/muuv/me, hostPort=43147, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/shard-1-001/cores}
[junit4] 2> 2130400 ERROR (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2130400 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.2.0
[junit4] 2> 2130400 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2130400 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2130400 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-09T20:56:40.943Z
[junit4] 2> 2130401 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2130401 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/shard-1-001/solr.xml
[junit4] 2> 2130403 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2130403 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2130404 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@501633a1, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2130405 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40351/solr
[junit4] 2> 2130412 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2130413 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2130413 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43147_muuv%2Fme
[junit4] 2> 2130423 INFO (zkCallback-3783-thread-2-processing-n:127.0.0.1:42873_muuv%2Fme) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2130423 INFO (zkCallback-3791-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2130439 INFO (zkCallback-3796-thread-1-processing-n:127.0.0.1:43147_muuv%2Fme) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2130461 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@501633a1
[junit4] 2> 2130467 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@501633a1
[junit4] 2> 2130468 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@501633a1
[junit4] 2> 2130469 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/shard-1-001/cores
[junit4] 2> 2130484 INFO (qtp494783358-18299) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:43147_muuv%252Fme&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2130485 INFO (OverseerCollectionConfigSetProcessor-98976464762699780-127.0.0.1:42873_muuv%2Fme-n_0000000000) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000002 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2130485 INFO (OverseerThreadFactory-7327-thread-3-processing-n:127.0.0.1:42873_muuv%2Fme) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:43147_muuv%2Fme for creating new replica
[junit4] 2> 2130486 INFO (SocketProxy-Acceptor-43147) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=56500,localport=43147], receiveBufferSize:531000
[junit4] 2> 2130486 INFO (SocketProxy-Acceptor-43147) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=39617,localport=35082], receiveBufferSize=530904
[junit4] 2> 2130487 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 2130487 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2130602 INFO (zkCallback-3796-thread-1-processing-n:127.0.0.1:43147_muuv%2Fme) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2131514 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.2.0
[junit4] 2> 2131583 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.IndexSchema [collection1_shard2_replica_t41] Schema name=test
[junit4] 2> 2131670 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2131676 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_t41' using configuration from collection collection1, trusted=true
[junit4] 2> 2131676 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_t41' (registry 'solr.core.collection1.shard2.replica_t41') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@501633a1
[junit4] 2> 2131676 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2131676 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SolrCore [[collection1_shard2_replica_t41] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/shard-1-001/cores/collection1_shard2_replica_t41], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/shard-1-001/cores/collection1_shard2_replica_t41/data/]
[junit4] 2> 2131678 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=16, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 2131679 WARN (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2131700 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2131700 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2131701 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2131701 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2131702 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=98.857421875, floorSegmentMB=1.287109375, forceMergeDeletesPctAllowed=5.777064652578674, segmentsPerTier=36.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5614771407648974
[junit4] 2> 2131702 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.SolrIndexSearcher Opening [Searcher@5f971277[collection1_shard2_replica_t41] main]
[junit4] 2> 2131703 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2131703 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2131704 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2131705 INFO (searcherExecutor-7341-thread-1-processing-n:127.0.0.1:43147_muuv%2Fme x:collection1_shard2_replica_t41 s:shard2 c:collection1) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SolrCore [collection1_shard2_replica_t41] Registered new searcher Searcher@5f971277[collection1_shard2_replica_t41] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2131705 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1583623440693198848
[junit4] 2> 2131708 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2131708 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2131708 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:43147/muuv/me/collection1_shard2_replica_t41/
[junit4] 2> 2131708 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2131708 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.SyncStrategy http://127.0.0.1:43147/muuv/me/collection1_shard2_replica_t41/ has no replicas
[junit4] 2> 2131708 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2131708 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ZkController collection1_shard2_replica_t41 stopping background replication from leader
[junit4] 2> 2131709 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:43147/muuv/me/collection1_shard2_replica_t41/ shard2
[junit4] 2> 2131810 INFO (zkCallback-3796-thread-1-processing-n:127.0.0.1:43147_muuv%2Fme) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2131860 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2131861 INFO (qtp703113789-18350) [n:127.0.0.1:43147_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t41] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t41&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1374
[junit4] 2> 2131862 INFO (qtp494783358-18299) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:43147_muuv%252Fme&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2} status=0 QTime=1378
[junit4] 2> 2131928 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 2 in directory /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/shard-2-001 of type TLOG
[junit4] 2> 2131928 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2131929 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@271eb14f{/muuv/me,null,AVAILABLE}
[junit4] 2> 2131929 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1bde17a{HTTP/1.1,[http/1.1]}{127.0.0.1:45587}
[junit4] 2> 2131929 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.e.j.s.Server Started @2134462ms
[junit4] 2> 2131929 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/tempDir-001/jetty2, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/muuv/me, hostPort=41483, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/shard-2-001/cores}
[junit4] 2> 2131929 ERROR (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2131929 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.2.0
[junit4] 2> 2131929 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2131929 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2131929 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-09T20:56:42.472Z
[junit4] 2> 2131931 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2131931 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/shard-2-001/solr.xml
[junit4] 2> 2131933 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2131933 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2131934 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@501633a1, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2131936 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40351/solr
[junit4] 2> 2131952 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:41483_muuv%2Fme ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2131953 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:41483_muuv%2Fme ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2131955 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:41483_muuv%2Fme ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41483_muuv%2Fme
[junit4] 2> 2131955 INFO (zkCallback-3796-thread-1-processing-n:127.0.0.1:43147_muuv%2Fme) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2131955 INFO (zkCallback-3783-thread-2-processing-n:127.0.0.1:42873_muuv%2Fme) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2131958 INFO (zkCallback-3791-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2131961 INFO (zkCallback-3802-thread-1-processing-n:127.0.0.1:41483_muuv%2Fme) [n:127.0.0.1:41483_muuv%2Fme ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 2132012 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:41483_muuv%2Fme ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@501633a1
[junit4] 2> 2132023 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:41483_muuv%2Fme ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@501633a1
[junit4] 2> 2132024 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:41483_muuv%2Fme ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@501633a1
[junit4] 2> 2132025 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:41483_muuv%2Fme ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/shard-2-001/cores
[junit4] 2> 2132049 INFO (SocketProxy-Acceptor-41483) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=43998,localport=41483], receiveBufferSize:531000
[junit4] 2> 2132050 INFO (SocketProxy-Acceptor-41483) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=45587,localport=50140], receiveBufferSize=530904
[junit4] 2> 2132050 INFO (qtp1774234511-18387) [n:127.0.0.1:41483_muuv%2Fme ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:41483_muuv%252Fme&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2132051 INFO (OverseerCollectionConfigSetProcessor-98976464762699780-127.0.0.1:42873_muuv%2Fme-n_0000000000) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000004 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2132051 INFO (OverseerThreadFactory-7327-thread-4-processing-n:127.0.0.1:42873_muuv%2Fme) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:41483_muuv%2Fme for creating new replica
[junit4] 2> 2132052 INFO (SocketProxy-Acceptor-41483) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=44002,localport=41483], receiveBufferSize:531000
[junit4] 2> 2132054 INFO (SocketProxy-Acceptor-41483) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=45587,localport=50144], receiveBufferSize=530904
[junit4] 2> 2132054 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 2132054 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2132156 INFO (zkCallback-3802-thread-1-processing-n:127.0.0.1:41483_muuv%2Fme) [n:127.0.0.1:41483_muuv%2Fme ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2132156 INFO (zkCallback-3796-thread-1-processing-n:127.0.0.1:43147_muuv%2Fme) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2133065 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.2.0
[junit4] 2> 2133075 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.s.IndexSchema [collection1_shard1_replica_t43] Schema name=test
[junit4] 2> 2133191 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2133198 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_t43' using configuration from collection collection1, trusted=true
[junit4] 2> 2133199 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard1.replica_t43' (registry 'solr.core.collection1.shard1.replica_t43') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@501633a1
[junit4] 2> 2133199 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2133199 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SolrCore [[collection1_shard1_replica_t43] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/shard-2-001/cores/collection1_shard1_replica_t43], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/shard-2-001/cores/collection1_shard1_replica_t43/data/]
[junit4] 2> 2133201 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=16, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 2133204 WARN (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2133237 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2133237 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2133239 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2133239 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2133240 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=98.857421875, floorSegmentMB=1.287109375, forceMergeDeletesPctAllowed=5.777064652578674, segmentsPerTier=36.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5614771407648974
[junit4] 2> 2133240 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.s.SolrIndexSearcher Opening [Searcher@fad5ca6[collection1_shard1_replica_t43] main]
[junit4] 2> 2133241 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2133241 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2133242 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2133243 INFO (searcherExecutor-7352-thread-1-processing-n:127.0.0.1:41483_muuv%2Fme x:collection1_shard1_replica_t43 s:shard1 c:collection1) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SolrCore [collection1_shard1_replica_t43] Registered new searcher Searcher@fad5ca6[collection1_shard1_replica_t43] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2133243 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1583623442305908736
[junit4] 2> 2133245 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2133245 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2133245 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:41483/muuv/me/collection1_shard1_replica_t43/
[junit4] 2> 2133245 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2133245 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.SyncStrategy http://127.0.0.1:41483/muuv/me/collection1_shard1_replica_t43/ has no replicas
[junit4] 2> 2133246 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2133246 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ZkController collection1_shard1_replica_t43 stopping background replication from leader
[junit4] 2> 2133247 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:41483/muuv/me/collection1_shard1_replica_t43/ shard1
[junit4] 2> 2133348 INFO (zkCallback-3802-thread-1-processing-n:127.0.0.1:41483_muuv%2Fme) [n:127.0.0.1:41483_muuv%2Fme ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2133348 INFO (zkCallback-3796-thread-1-processing-n:127.0.0.1:43147_muuv%2Fme) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [3])
[junit4] 2> 2133397 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2133398 INFO (qtp1774234511-18389) [n:127.0.0.1:41483_muuv%2Fme c:collection1 s:shard1 x:collection1_shard1_replica_t43] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard1_replica_t43&action=CREATE&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1344
[junit4] 2> 2133400 INFO (qtp1774234511-18387) [n:127.0.0.1:41483_muuv%2Fme ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:41483_muuv%252Fme&action=ADDREPLICA&collection=collection1&shard=shard1&type=TLOG&wt=javabin&version=2} status=0 QTime=1349
[junit4] 2> 2133457 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.AbstractFullDistribZkTestBase create jetty 3 in directory /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/shard-3-001 of type TLOG
[junit4] 2> 2133457 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2133459 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@42dbd745{/muuv/me,null,AVAILABLE}
[junit4] 2> 2133459 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@11989139{HTTP/1.1,[http/1.1]}{127.0.0.1:40085}
[junit4] 2> 2133459 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.e.j.s.Server Started @2135992ms
[junit4] 2> 2133459 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/tempDir-001/jetty3, replicaType=TLOG, solrconfig=solrconfig.xml, hostContext=/muuv/me, hostPort=44073, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/shard-3-001/cores}
[junit4] 2> 2133459 ERROR (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2133459 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.2.0
[junit4] 2> 2133459 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2133459 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2133459 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-09T20:56:44.002Z
[junit4] 2> 2133461 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2133461 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/shard-3-001/solr.xml
[junit4] 2> 2133464 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2133464 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2133465 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@501633a1, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2133467 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40351/solr
[junit4] 2> 2133472 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:44073_muuv%2Fme ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 2133472 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:44073_muuv%2Fme ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2133473 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:44073_muuv%2Fme ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44073_muuv%2Fme
[junit4] 2> 2133473 INFO (zkCallback-3791-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2133473 INFO (zkCallback-3802-thread-1-processing-n:127.0.0.1:41483_muuv%2Fme) [n:127.0.0.1:41483_muuv%2Fme ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2133473 INFO (zkCallback-3796-thread-1-processing-n:127.0.0.1:43147_muuv%2Fme) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2133474 INFO (zkCallback-3783-thread-2-processing-n:127.0.0.1:42873_muuv%2Fme) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2133474 INFO (zkCallback-3808-thread-1-processing-n:127.0.0.1:44073_muuv%2Fme) [n:127.0.0.1:44073_muuv%2Fme ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (3) -> (4)
[junit4] 2> 2133573 INFO (zkCallback-3796-thread-1-processing-n:127.0.0.1:43147_muuv%2Fme) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2133574 INFO (zkCallback-3802-thread-1-processing-n:127.0.0.1:41483_muuv%2Fme) [n:127.0.0.1:41483_muuv%2Fme ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2133580 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:44073_muuv%2Fme ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@501633a1
[junit4] 2> 2133590 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:44073_muuv%2Fme ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@501633a1
[junit4] 2> 2133590 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:44073_muuv%2Fme ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@501633a1
[junit4] 2> 2133592 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [n:127.0.0.1:44073_muuv%2Fme ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/shard-3-001/cores
[junit4] 2> 2133626 INFO (qtp1774234511-18388) [n:127.0.0.1:41483_muuv%2Fme ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :addreplica with params node=127.0.0.1:44073_muuv%252Fme&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2133627 INFO (OverseerCollectionConfigSetProcessor-98976464762699780-127.0.0.1:42873_muuv%2Fme-n_0000000000) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000006 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2133627 INFO (OverseerThreadFactory-7327-thread-5-processing-n:127.0.0.1:42873_muuv%2Fme) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.AddReplicaCmd Node Identified 127.0.0.1:44073_muuv%2Fme for creating new replica
[junit4] 2> 2133628 INFO (SocketProxy-Acceptor-44073) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=42244,localport=44073], receiveBufferSize:531000
[junit4] 2> 2133629 INFO (SocketProxy-Acceptor-44073) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=40085,localport=42698], receiveBufferSize=530904
[junit4] 2> 2133629 INFO (qtp522694994-18421) [n:127.0.0.1:44073_muuv%2Fme ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 2133629 INFO (qtp522694994-18421) [n:127.0.0.1:44073_muuv%2Fme ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2133731 INFO (zkCallback-3802-thread-1-processing-n:127.0.0.1:41483_muuv%2Fme) [n:127.0.0.1:41483_muuv%2Fme ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2133731 INFO (zkCallback-3808-thread-1-processing-n:127.0.0.1:44073_muuv%2Fme) [n:127.0.0.1:44073_muuv%2Fme ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2133731 INFO (zkCallback-3796-thread-1-processing-n:127.0.0.1:43147_muuv%2Fme) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2134644 INFO (qtp522694994-18421) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.2.0
[junit4] 2> 2134654 INFO (qtp522694994-18421) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.s.IndexSchema [collection1_shard2_replica_t45] Schema name=test
[junit4] 2> 2134798 INFO (qtp522694994-18421) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 2134808 INFO (qtp522694994-18421) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_t45' using configuration from collection collection1, trusted=true
[junit4] 2> 2134808 INFO (qtp522694994-18421) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1.shard2.replica_t45' (registry 'solr.core.collection1.shard2.replica_t45') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@501633a1
[junit4] 2> 2134808 INFO (qtp522694994-18421) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2134808 INFO (qtp522694994-18421) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.c.SolrCore [[collection1_shard2_replica_t45] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/shard-3-001/cores/collection1_shard2_replica_t45], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001/shard-3-001/cores/collection1_shard2_replica_t45/data/]
[junit4] 2> 2134810 INFO (qtp522694994-18421) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=16, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
[junit4] 2> 2134812 WARN (qtp522694994-18421) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 2134841 INFO (qtp522694994-18421) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2134841 INFO (qtp522694994-18421) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2134842 INFO (qtp522694994-18421) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2134843 INFO (qtp522694994-18421) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2134843 INFO (qtp522694994-18421) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=40, maxMergeAtOnceExplicit=40, maxMergedSegmentMB=98.857421875, floorSegmentMB=1.287109375, forceMergeDeletesPctAllowed=5.777064652578674, segmentsPerTier=36.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.5614771407648974
[junit4] 2> 2134844 INFO (qtp522694994-18421) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.s.SolrIndexSearcher Opening [Searcher@4165819[collection1_shard2_replica_t45] main]
[junit4] 2> 2134844 INFO (qtp522694994-18421) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 2134844 INFO (qtp522694994-18421) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 2134845 INFO (qtp522694994-18421) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2134846 INFO (searcherExecutor-7363-thread-1-processing-n:127.0.0.1:44073_muuv%2Fme x:collection1_shard2_replica_t45 s:shard2 c:collection1) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.c.SolrCore [collection1_shard2_replica_t45] Registered new searcher Searcher@4165819[collection1_shard2_replica_t45] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2134846 INFO (qtp522694994-18421) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1583623443986776064
[junit4] 2> 2134848 INFO (qtp522694994-18421) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.c.ZkController Core needs to recover:collection1_shard2_replica_t45
[junit4] 2> 2134849 INFO (updateExecutor-3805-thread-1-processing-n:127.0.0.1:44073_muuv%2Fme x:collection1_shard2_replica_t45 s:shard2 c:collection1) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_t45] o.a.s.u.DefaultSolrCoreState Running recovery
[junit4] 2> 2134849 INFO (qtp522694994-18421) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 x:collection1_shard2_replica_t45] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&name=collection1_shard2_replica_t45&action=CREATE&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1220
[junit4] 2> 2134850 INFO (recoveryExecutor-3806-thread-1-processing-n:127.0.0.1:44073_muuv%2Fme x:collection1_shard2_replica_t45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_t45] o.a.s.c.RecoveryStrategy Starting recovery process. recoveringAfterStartup=true
[junit4] 2> 2134850 INFO (recoveryExecutor-3806-thread-1-processing-n:127.0.0.1:44073_muuv%2Fme x:collection1_shard2_replica_t45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_t45] o.a.s.c.RecoveryStrategy ###### startupVersions=[[]]
[junit4] 2> 2134850 INFO (recoveryExecutor-3806-thread-1-processing-n:127.0.0.1:44073_muuv%2Fme x:collection1_shard2_replica_t45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_t45] o.a.s.c.ZkController collection1_shard2_replica_t45 stopping background replication from leader
[junit4] 2> 2134850 INFO (recoveryExecutor-3806-thread-1-processing-n:127.0.0.1:44073_muuv%2Fme x:collection1_shard2_replica_t45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_t45] o.a.s.c.RecoveryStrategy Begin buffering updates. core=[collection1_shard2_replica_t45]
[junit4] 2> 2134851 INFO (recoveryExecutor-3806-thread-1-processing-n:127.0.0.1:44073_muuv%2Fme x:collection1_shard2_replica_t45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_t45] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 2134851 INFO (recoveryExecutor-3806-thread-1-processing-n:127.0.0.1:44073_muuv%2Fme x:collection1_shard2_replica_t45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_t45] o.a.s.c.RecoveryStrategy Publishing state of core [collection1_shard2_replica_t45] as recovering, leader is [http://127.0.0.1:43147/muuv/me/collection1_shard2_replica_t41/] and I am [http://127.0.0.1:44073/muuv/me/collection1_shard2_replica_t45/]
[junit4] 2> 2134852 INFO (recoveryExecutor-3806-thread-1-processing-n:127.0.0.1:44073_muuv%2Fme x:collection1_shard2_replica_t45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_t45] o.a.s.c.RecoveryStrategy Sending prep recovery command to [http://127.0.0.1:43147/muuv/me]; [WaitForState: action=PREPRECOVERY&core=collection1_shard2_replica_t41&nodeName=127.0.0.1:44073_muuv%252Fme&coreNodeName=core_node46&state=recovering&checkLive=true&onlyIfLeader=true&onlyIfLeaderActive=true]
[junit4] 2> 2134852 INFO (SocketProxy-Acceptor-43147) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=56568,localport=43147], receiveBufferSize:531000
[junit4] 2> 2134856 INFO (qtp1774234511-18388) [n:127.0.0.1:41483_muuv%2Fme ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={node=127.0.0.1:44073_muuv%252Fme&action=ADDREPLICA&collection=collection1&shard=shard2&type=TLOG&wt=javabin&version=2} status=0 QTime=1230
[junit4] 2> 2134856 INFO (SocketProxy-Acceptor-43147) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=39617,localport=35150], receiveBufferSize=530904
[junit4] 2> 2134857 INFO (qtp703113789-18351) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.h.a.PrepRecoveryOp Going to wait for coreNodeName: core_node46, state: recovering, checkLive: true, onlyIfLeader: true, onlyIfLeaderActive: true, maxTime: 183 s
[junit4] 2> 2134858 INFO (qtp703113789-18351) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1_shard2_replica_t41, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=down, localState=active, nodeName=127.0.0.1:44073_muuv%2Fme, coreNodeName=core_node46, onlyIfActiveCheckResult=false, nodeProps: core_node46:{"core":"collection1_shard2_replica_t45","base_url":"http://127.0.0.1:44073/muuv/me","node_name":"127.0.0.1:44073_muuv%2Fme","state":"down","type":"TLOG"}
[junit4] 2> 2134860 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.SolrTestCaseJ4 ###Starting test
[junit4] 2> 2134860 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Wait for recoveries to finish - wait 30000 for each attempt
[junit4] 2> 2134860 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):30000
[junit4] 2> 2134953 INFO (zkCallback-3796-thread-1-processing-n:127.0.0.1:43147_muuv%2Fme) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2134953 INFO (zkCallback-3808-thread-1-processing-n:127.0.0.1:44073_muuv%2Fme) [n:127.0.0.1:44073_muuv%2Fme ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2134953 INFO (zkCallback-3802-thread-1-processing-n:127.0.0.1:41483_muuv%2Fme) [n:127.0.0.1:41483_muuv%2Fme ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [4])
[junit4] 2> 2135629 INFO (OverseerCollectionConfigSetProcessor-98976464762699780-127.0.0.1:42873_muuv%2Fme-n_0000000000) [n:127.0.0.1:42873_muuv%2Fme ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000008 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2135859 INFO (qtp703113789-18351) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.h.a.PrepRecoveryOp In WaitForState(recovering): collection=collection1, shard=shard2, thisCore=collection1_shard2_replica_t41, leaderDoesNotNeedRecovery=false, isLeader? true, live=true, checkLive=true, currentState=recovering, localState=active, nodeName=127.0.0.1:44073_muuv%2Fme, coreNodeName=core_node46, onlyIfActiveCheckResult=false, nodeProps: core_node46:{"core":"collection1_shard2_replica_t45","base_url":"http://127.0.0.1:44073/muuv/me","node_name":"127.0.0.1:44073_muuv%2Fme","state":"recovering","type":"TLOG"}
[junit4] 2> 2135859 INFO (qtp703113789-18351) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.h.a.PrepRecoveryOp Waited coreNodeName: core_node46, state: recovering, checkLive: true, onlyIfLeader: true for: 1 seconds.
[junit4] 2> 2135859 INFO (qtp703113789-18351) [n:127.0.0.1:43147_muuv%2Fme ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={nodeName=127.0.0.1:44073_muuv%252Fme&onlyIfLeaderActive=true&core=collection1_shard2_replica_t41&coreNodeName=core_node46&action=PREPRECOVERY&checkLive=true&state=recovering&onlyIfLeader=true&wt=javabin&version=2} status=0 QTime=1001
[junit4] 2> 2136360 INFO (recoveryExecutor-3806-thread-1-processing-n:127.0.0.1:44073_muuv%2Fme x:collection1_shard2_replica_t45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_t45] o.a.s.c.RecoveryStrategy Starting Replication Recovery.
[junit4] 2> 2136360 INFO (recoveryExecutor-3806-thread-1-processing-n:127.0.0.1:44073_muuv%2Fme x:collection1_shard2_replica_t45 s:shard2 c:collection1 r:core_node46) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_t45] o.a.s.c.RecoveryStrategy Attempting to replicate from [http://127.0.0.1:43147/muuv/me/collection1_shard2_replica_t41/].
[junit4] 2> 2136360 INFO (SocketProxy-Acceptor-43147) [ ] o.a.s.c.SocketProxy accepted Socket[addr=/127.0.0.1,port=56582,localport=43147], receiveBufferSize:531000
[junit4] 2> 2136361 INFO (SocketProxy-Acceptor-43147) [ ] o.a.s.c.SocketProxy proxy connection Socket[addr=/127.0.0.1,port=39617,localport=35164], receiveBufferSize=530904
[junit4] 2> 2136361 INFO (qtp703113789-18
[...truncated too long message...]
Manager Closing metric reporters for registry=solr.core.collection1.shard2.replica_t45, tag=1830327507
[junit4] 2> 2206495 INFO (zkCallback-3808-thread-2-processing-n:127.0.0.1:44073_muuv%2Fme) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_t45] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@154d18ff: rootName = null, domain = solr.core.collection1.shard2.replica_t45, service url = null, agent id = null] for registry solr.core.collection1.shard2.replica_t45 / com.codahale.metrics.MetricRegistry@75bb3f6a
[junit4] 2> 2206503 INFO (zkCallback-3808-thread-2-processing-n:127.0.0.1:44073_muuv%2Fme) [n:127.0.0.1:44073_muuv%2Fme c:collection1 s:shard2 r:core_node46 x:collection1_shard2_replica_t45] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard2.leader, tag=1830327507
[junit4] 2> 2206516 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@11989139{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2206516 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@42dbd745{/muuv/me,null,UNAVAILABLE}
[junit4] 2> 2206518 ERROR (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2206521 INFO (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40351 40351
[junit4] 2> 2211552 INFO (Thread-3881) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40351 40351
[junit4] 2> 2211552 WARN (Thread-3881) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/aliases.json
[junit4] 2> 4 /solr/security.json
[junit4] 2> 4 /solr/configs/conf1
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/clusterprops.json
[junit4] 2> 6 /solr/clusterstate.json
[junit4] 2> 4 /solr/autoscaling.json
[junit4] 2> 3 /solr/collections/collection1/state.json
[junit4] 2> 3 /solr/collections/collMinRf_1x3/state.json
[junit4] 2> 2 /solr/collections/collMinRf_1x3/leader_elect/shard1/election/98976464762699791-core_node6-n_0000000000
[junit4] 2> 2 /solr/collections/c8n_1x2/state.json
[junit4] 2> 2 /solr/collections/c8n_crud_1x2/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/live_nodes
[junit4] 2> 6 /solr/collections
[junit4] 2> 4 /solr/autoscaling/events/.auto_add_replicas
[junit4] 2> 4 /solr/overseer/collection-queue-work
[junit4] 2> 4 /solr/overseer/queue-work
[junit4] 2> 3 /solr/overseer/queue
[junit4] 2>
[junit4] 2> 2211553 WARN (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.SocketProxy Closing 27 connections to: http://127.0.0.1:44073/muuv/me, target: http://127.0.0.1:40085/muuv/me
[junit4] 2> 2211553 WARN (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.SocketProxy Closing 0 connections to: http://127.0.0.1:41483/muuv/me, target: http://127.0.0.1:45587/muuv/me
[junit4] 2> 2211553 WARN (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.SocketProxy Closing 5 connections to: http://127.0.0.1:43147/muuv/me, target: http://127.0.0.1:39617/muuv/me
[junit4] 2> 2211553 WARN (TEST-HttpPartitionTest.test-seed#[DC06EE4BF292881F]) [ ] o.a.s.c.SocketProxy Closing 7 connections to: http://127.0.0.1:42873/muuv/me, target: http://127.0.0.1:46587/muuv/me
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=HttpPartitionTest -Dtests.method=test -Dtests.seed=DC06EE4BF292881F -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=bg-BG -Dtests.timezone=Etc/GMT+4 -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] FAILURE 84.1s J0 | HttpPartitionTest.test <<<
[junit4] > Throwable #1: java.lang.AssertionError: max version bucket seed not updated after recovery!
[junit4] > at __randomizedtesting.SeedInfo.seed([DC06EE4BF292881F:5452D1915C6EE5E7]:0)
[junit4] > at org.apache.solr.cloud.HttpPartitionTest.testRf2(HttpPartitionTest.java:377)
[junit4] > at org.apache.solr.cloud.HttpPartitionTest.test(HttpPartitionTest.java:132)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.HttpPartitionTest_DC06EE4BF292881F-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {multiDefault=PostingsFormat(name=Memory), a_t=PostingsFormat(name=MockRandom), id=FSTOrd50, text=PostingsFormat(name=Asserting)}, docValues:{range_facet_l_dv=DocValuesFormat(name=Memory), _version_=DocValuesFormat(name=Lucene70), range_facet_i_dv=DocValuesFormat(name=Lucene70), intDvoDefault=DocValuesFormat(name=Lucene70), timestamp=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=1702, maxMBSortInHeap=5.961377353374648, sim=RandomSimilarity(queryNorm=true): {}, locale=bg-BG, timezone=Etc/GMT+4
[junit4] 2> NOTE: Linux 4.10.0-37-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=8,threads=1,free=290563376,total=536870912
[junit4] 2> NOTE: All tests run in this JVM: [BigEndianAscendingWordSerializerTest, TestManagedStopFilterFactory, LargeFieldTest, TestFieldSortValues, URLClassifyProcessorTest, TestReloadDeadlock, TestFastOutputStream, TestOnReconnectListenerSupport, FileUtilsTest, CursorPagingTest, TestSha256AuthenticationProvider, TestFreeTextSuggestions, AnalyticsMergeStrategyTest, HdfsUnloadDistributedZkTest, TestSort, BlockDirectoryTest, V2StandaloneTest, TestReloadAndDeleteDocs, TestRebalanceLeaders, TestSolrDeletionPolicy1, CloudMLTQParserTest, SearchHandlerTest, TestSolrFieldCacheBean, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestExportWriter, TestAuthorizationFramework, SolrCoreTest, TestDocBasedVersionConstraints, TestNumericRangeQuery64, TestFieldCacheVsDocValues, TestSolrJ, TestRandomFaceting, TestBinaryField, PrimitiveFieldTypeTest, TestDFISimilarityFactory, SortSpecParsingTest, DeleteInactiveReplicaTest, SpatialFilterTest, TestUpdate, HdfsAutoAddReplicasIntegrationTest, SpellCheckCollatorTest, HdfsLockFactoryTest, LeaderInitiatedRecoveryOnShardRestartTest, TestIBSimilarityFactory, ExitableDirectoryReaderTest, AtomicUpdatesTest, LeaderFailoverAfterPartitionTest, TestSubQueryTransformerDistrib, TestShortCircuitedRequests, TestPolicyCloud, SparseHLLTest, SolrMetricManagerTest, CacheHeaderTest, SolrShardReporterTest, DistributedDebugComponentTest, QueryParsingTest, TestBlobHandler, TestReplicationHandler, TestPhraseSuggestions, TestTlogReplica, BasicAuthIntegrationTest, VersionInfoTest, TestClusterStateMutator, CleanupOldIndexTest, TestSQLHandler, TestFieldCacheReopen, TermsComponentTest, SliceStateTest, SimpleFacetsTest, BadCopyFieldTest, TestFieldResource, InfixSuggestersTest, TestMacroExpander, SolrJmxReporterTest, BlobRepositoryCloudTest, TestSmileRequest, MetricsHandlerTest, NumberUtilsTest, DistributedQueryElevationComponentTest, TestExactSharedStatsCache, TestDynamicFieldCollectionResource, TestHashQParserPlugin, DirectUpdateHandlerOptimizeTest, DistributedMLTComponentTest, TestLuceneIndexBackCompat, FullHLLTest, TestJoin, TestJsonFacetRefinement, TestSchemaVersionResource, DocValuesTest, TestLeaderElectionWithEmptyReplica, HighlighterMaxOffsetTest, TestChildDocTransformer, TestCodecSupport, HdfsBasicDistributedZk2Test, TestRTimerTree, TestSweetSpotSimilarityFactory, TestSuggestSpellingConverter, JvmMetricsTest, TestSolrConfigHandlerConcurrent, TestSolrIndexConfig, TestDFRSimilarityFactory, SegmentsInfoRequestHandlerTest, DistributedFacetPivotSmallTest, StatelessScriptUpdateProcessorFactoryTest, TestConfigSetsAPI, TestWordDelimiterFilterFactory, ClassificationUpdateProcessorTest, TestConfig, SolrPluginUtilsTest, TestLocalFSCloudBackupRestore, AnalysisAfterCoreReloadTest, UpdateParamsTest, TestInPlaceUpdatesDistrib, TestCollectionAPIs, SaslZkACLProviderTest, TestSolrCloudWithDelegationTokens, CurrencyFieldTypeTest, TermVectorComponentTest, TestRandomDVFaceting, ConnectionManagerTest, TestIndexingPerformance, BasicZkTest, LeaderElectionIntegrationTest, RecoveryZkTest, ClusterStateUpdateTest, ZkSolrClientTest, ShardRoutingCustomTest, TestDistributedGrouping, TestFaceting, TestRecovery, TermVectorComponentDistributedTest, ZkControllerTest, TestStressVersions, StatsComponentTest, TestGroupingSearch, TestFiltering, SoftAutoCommitTest, PreAnalyzedFieldTest, DateFieldTest, RAMDirectoryFactoryTest, TestLRUCache, TestUtils, TestDocumentBuilder, ZkNodePropsTest, TestRTGBase, DistributedIntervalFacetingTest, SolrTestCaseJ4Test, TestCrossCoreJoin, TestCursorMarkWithoutUniqueKey, TestDistributedMissingSort, TestHighlightDedupGrouping, TestSimpleTrackingShardHandler, TestEmbeddedSolrServerConstructors, TestEmbeddedSolrServerSchemaAPI, ActionThrottleTest, AddReplicaTest, AssignBackwardCompatibilityTest, ChaosMonkeySafeLeaderWithPullReplicasTest, CollectionTooManyReplicasTest, CollectionsAPISolrJTest, DeleteShardTest, DistribJoinFromCollectionTest, DistributedQueueTest, HttpPartitionTest]
[junit4] Completed [513/748 (1!)] on J0 in 84.21s, 1 test, 1 failure <<< FAILURES!
[...truncated 48091 lines...]
[JENKINS] Lucene-Solr-7.x-Linux (32bit/jdk1.8.0_144) - Build # 787
- Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/787/
Java: 32bit/jdk1.8.0_144 -server -XX:+UseConcMarkSweepGC
3 tests failed.
FAILED: org.apache.solr.client.solrj.impl.CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale
Error Message:
Error from server at https://127.0.0.1:36949/solr/stale_state_test_col_shard1_replica_n1: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 404 </title> </head> <body> <h2>HTTP ERROR: 404</h2> <p>Problem accessing /solr/stale_state_test_col_shard1_replica_n1/update. Reason: <pre> Can not find: /solr/stale_state_test_col_shard1_replica_n1/update</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.20.v20170531</a><hr/> </body> </html>
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:36949/solr/stale_state_test_col_shard1_replica_n1: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 </title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /solr/stale_state_test_col_shard1_replica_n1/update. Reason:
<pre> Can not find: /solr/stale_state_test_col_shard1_replica_n1/update</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.20.v20170531</a><hr/>
</body>
</html>
at __randomizedtesting.SeedInfo.seed([52C158DA1A9AA452:E6F0C032F973D27E]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:559)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1016)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:883)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
at org.apache.solr.client.solrj.impl.CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale(CloudSolrClientTest.java:844)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.PeerSyncReplicationTest.test
Error Message:
timeout waiting to see all nodes active
Stack Trace:
java.lang.AssertionError: timeout waiting to see all nodes active
at __randomizedtesting.SeedInfo.seed([1D36E40FDB869193:9562DBD5757AFC6B]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.PeerSyncReplicationTest.waitTillNodesActive(PeerSyncReplicationTest.java:367)
at org.apache.solr.cloud.PeerSyncReplicationTest.bringUpDeadNodeAndEnsureNoReplication(PeerSyncReplicationTest.java:310)
at org.apache.solr.cloud.PeerSyncReplicationTest.test(PeerSyncReplicationTest.java:170)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:993)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.UnloadDistributedZkTest.test
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:37935
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:37935
at __randomizedtesting.SeedInfo.seed([1D36E40FDB869193:9562DBD5757AFC6B]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:883)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:314)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
... 43 more
Build Log:
[...truncated 12833 lines...]
[junit4] Suite: org.apache.solr.cloud.UnloadDistributedZkTest
[junit4] 2> 1299354 INFO (SUITE-UnloadDistributedZkTest-seed#[1D36E40FDB869193]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.UnloadDistributedZkTest_1D36E40FDB869193-001/init-core-data-001
[junit4] 2> 1299355 WARN (SUITE-UnloadDistributedZkTest-seed#[1D36E40FDB869193]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=17 numCloses=17
[junit4] 2> 1299355 INFO (SUITE-UnloadDistributedZkTest-seed#[1D36E40FDB869193]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 1299356 INFO (SUITE-UnloadDistributedZkTest-seed#[1D36E40FDB869193]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=https://issues.apache.org/jira/browse/SOLR-5776)
[junit4] 2> 1299356 INFO (SUITE-UnloadDistributedZkTest-seed#[1D36E40FDB869193]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 1299358 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1299358 INFO (Thread-2503) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1299358 INFO (Thread-2503) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1299359 ERROR (Thread-2503) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1299458 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.ZkTestServer start zk server on port:40619
[junit4] 2> 1299463 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 1299464 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 1299464 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 1299465 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 1299465 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 1299466 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 1299466 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 1299466 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 1299467 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 1299467 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 1299467 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 1299468 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use TLOG replicas unless explicitly asked otherwise
[junit4] 2> 1299536 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 1299537 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1342371{/,null,AVAILABLE}
[junit4] 2> 1299537 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@db5f9b{HTTP/1.1,[http/1.1]}{127.0.0.1:๓๗๙๓๕}
[junit4] 2> 1299537 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.e.j.s.Server Started @๑๓๐๑๒๐๓ms
[junit4] 2> 1299537 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.UnloadDistributedZkTest_1D36E40FDB869193-001/tempDir-001/control/data, hostContext=/, hostPort=37935, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.UnloadDistributedZkTest_1D36E40FDB869193-001/control-001/cores}
[junit4] 2> 1299537 ERROR (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1299538 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.2.0
[junit4] 2> 1299538 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1299538 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 1299538 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-10T12:46:51.457Z
[junit4] 2> 1299544 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 1299544 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.UnloadDistributedZkTest_1D36E40FDB869193-001/control-001/solr.xml
[junit4] 2> 1299546 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 1299546 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 1299546 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@11e868c, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 1299549 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:40619/solr
[junit4] 2> 1299573 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [n:127.0.0.1:37935_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1299573 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [n:127.0.0.1:37935_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37935_
[junit4] 2> 1299574 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [n:127.0.0.1:37935_ ] o.a.s.c.Overseer Overseer (id=98980201187115012-127.0.0.1:37935_-n_0000000000) starting
[junit4] 2> 1299577 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [n:127.0.0.1:37935_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37935_
[junit4] 2> 1299578 INFO (zkCallback-2219-thread-1-processing-n:127.0.0.1:37935_) [n:127.0.0.1:37935_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1299737 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [n:127.0.0.1:37935_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11e868c
[junit4] 2> 1299743 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [n:127.0.0.1:37935_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11e868c
[junit4] 2> 1299744 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [n:127.0.0.1:37935_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11e868c
[junit4] 2> 1299745 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [n:127.0.0.1:37935_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.UnloadDistributedZkTest_1D36E40FDB869193-001/control-001/cores
[junit4] 2> 1299758 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1299758 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:40619/solr ready
[junit4] 2> 1299759 INFO (qtp10770406-11285) [n:127.0.0.1:37935_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:37935_&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1299760 INFO (OverseerThreadFactory-5267-thread-1-processing-n:127.0.0.1:37935_) [n:127.0.0.1:37935_ ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 1299863 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1299863 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 1299965 INFO (zkCallback-2219-thread-1-processing-n:127.0.0.1:37935_) [n:127.0.0.1:37935_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1300884 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.2.0
[junit4] 2> 1300897 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 1300960 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 1300967 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 1300967 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@11e868c
[junit4] 2> 1300967 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1300967 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.UnloadDistributedZkTest_1D36E40FDB869193-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.UnloadDistributedZkTest_1D36E40FDB869193-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 1300969 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=19, maxMergeAtOnceExplicit=32, maxMergedSegmentMB=20.650390625, floorSegmentMB=0.8720703125, forceMergeDeletesPctAllowed=2.1669962118140997, segmentsPerTier=45.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 1300971 WARN (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 1300995 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1300995 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1300997 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1300997 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1300998 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=32, maxMergeAtOnceExplicit=18, maxMergedSegmentMB=52.458984375, floorSegmentMB=0.607421875, forceMergeDeletesPctAllowed=9.863001346828645, segmentsPerTier=46.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3094712854034659
[junit4] 2> 1300998 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@cc5d08[control_collection_shard1_replica_n1] main]
[junit4] 2> 1300999 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 1300999 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 1301000 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 1301000 INFO (searcherExecutor-5270-thread-1-processing-n:127.0.0.1:37935_ x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@cc5d08[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1301000 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1583683220714553344
[junit4] 2> 1301003 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1301003 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1301003 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:37935/control_collection_shard1_replica_n1/
[junit4] 2> 1301003 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1301003 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:37935/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 1301003 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1301004 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:37935/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 1359821 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.ChaosMonkey monkey: stop jetty! 37935
[junit4] 2> 1359821 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=3186912
[junit4] 2> 1359821 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1359821 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1e87c18: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@62007f
[junit4] 2> 1359826 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1359826 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2173ce: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@ab5f39
[junit4] 2> 1359830 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1359831 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@f45276: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@c37e30
[junit4] 2> 1359832 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:37935_
[junit4] 2> 1359833 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1359833 INFO (zkCallback-2219-thread-2-processing-n:127.0.0.1:37935_) [n:127.0.0.1:37935_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1359833 INFO (coreCloseExecutor-5275-thread-1) [n:127.0.0.1:37935_ c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@70513a
[junit4] 2> 1359834 INFO (coreCloseExecutor-5275-thread-1) [n:127.0.0.1:37935_ c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.control_collection.shard1.replica_n1, tag=7360826
[junit4] 2> 1359834 INFO (coreCloseExecutor-5275-thread-1) [n:127.0.0.1:37935_ c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@8a1dba: rootName = null, domain = solr.core.control_collection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.control_collection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@16a1556
[junit4] 2> 1359842 INFO (coreCloseExecutor-5275-thread-1) [n:127.0.0.1:37935_ c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.control_collection.shard1.leader, tag=7360826
[junit4] 2> 1370588 INFO (ScheduledTrigger-5265-thread-1) [n:127.0.0.1:37935_ ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 1370592 INFO (AutoscalingActionExecutor-5266-thread-1-processing-n:127.0.0.1:37935_) [n:127.0.0.1:37935_ ] o.a.s.c.a.ExecutePlanAction No operations to execute for event: {
[junit4] 2> "id":"14f5ba48f1741e00T19t0ol0j3efrics28iit5b0hd",
[junit4] 2> "source":".auto_add_replicas",
[junit4] 2> "eventTime":1510318072504000000,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1510318072504000000],
[junit4] 2> "_enqueue_time_":1510318082506000000,
[junit4] 2> "nodeNames":["127.0.0.1:37935_"]}}
[junit4] 2> 1479761 ERROR (qtp10770406-11285) [n:127.0.0.1:37935_ ] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: create the collection time out:180s
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:312)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:244)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:222)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:737)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:718)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:499)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 1479762 INFO (qtp10770406-11285) [n:127.0.0.1:37935_ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:37935_&wt=javabin&version=2} status=500 QTime=180002
[junit4] 2> 1479762 ERROR (qtp10770406-11285) [n:127.0.0.1:37935_ ] o.a.s.s.HttpSolrCall null:org.apache.solr.common.SolrException: create the collection time out:180s
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:312)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:244)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:222)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:737)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:718)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:499)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 1899963 ERROR (OverseerThreadFactory-5267-thread-1-processing-n:127.0.0.1:37935_) [n:127.0.0.1:37935_ ] o.a.s.c.OverseerCollectionMessageHandler Error from shard: http://127.0.0.1:37935
[junit4] 2> org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:37935
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] 2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
[junit4] 2> at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:172)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.net.SocketTimeoutException: Read timed out
[junit4] 2> at java.net.SocketInputStream.socketRead0(Native Method)
[junit4] 2> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
[junit4] 2> at java.net.SocketInputStream.read(SocketInputStream.java:171)
[junit4] 2> at java.net.SocketInputStream.read(SocketInputStream.java:141)
[junit4] 2> at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
[junit4] 2> at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
[junit4] 2> at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
[junit4] 2> at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
[junit4] 2> at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
[junit4] 2> at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
[junit4] 2> at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
[junit4] 2> at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
[junit4] 2> at org.apache.solr.util.stats.InstrumentedHttpRequestExecutor.execute(InstrumentedHttpRequestExecutor.java:118)
[junit4] 2> at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
[junit4] 2> at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
[junit4] 2> at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
[junit4] 2> at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
[junit4] 2> at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
[junit4] 2> ... 12 more
[junit4] 2> 1899965 ERROR (OverseerThreadFactory-5267-thread-1-processing-n:127.0.0.1:37935_) [n:127.0.0.1:37935_ ] o.a.s.c.OverseerCollectionMessageHandler Cleaning up collection [control_collection].
[junit4] 2> 1899964 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.Overseer Overseer (id=98980201187115012-127.0.0.1:37935_-n_0000000000) closing
[junit4] 2> 1899965 INFO (OverseerStateUpdate-98980201187115012-127.0.0.1:37935_-n_0000000000) [n:127.0.0.1:37935_ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:37935_
[junit4] 2> 1899966 INFO (OverseerThreadFactory-5267-thread-1-processing-n:127.0.0.1:37935_) [n:127.0.0.1:37935_ ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
[junit4] 2> 1930004 INFO (zkCallback-2219-thread-4-processing-n:127.0.0.1:37935_) [n:127.0.0.1:37935_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1930006 ERROR (OverseerThreadFactory-5267-thread-1-processing-n:127.0.0.1:37935_) [n:127.0.0.1:37935_ ] o.a.s.c.OverseerCollectionMessageHandler Collection: control_collection operation: create failed:org.apache.solr.common.SolrException: Could not fully remove collection: control_collection
[junit4] 2> at org.apache.solr.cloud.DeleteCollectionCmd.call(DeleteCollectionCmd.java:109)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.cleanupCollection(OverseerCollectionMessageHandler.java:678)
[junit4] 2> at org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:302)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:232)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:464)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 1930013 INFO (OverseerThreadFactory-5267-thread-1-processing-n:127.0.0.1:37935_) [n:127.0.0.1:37935_ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1930014 WARN (OverseerAutoScalingTriggerThread-98980201187115012-127.0.0.1:37935_-n_0000000000) [n:127.0.0.1:37935_ ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1930017 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@db5f9b{HTTP/1.1,[http/1.1]}{127.0.0.1:๐}
[junit4] 2> 1930017 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1342371{/,null,UNAVAILABLE}
[junit4] 2> 1930022 ERROR (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController Error getting leader from zk
[junit4] 2> org.apache.solr.common.SolrException: No registered leader was found after waiting for 1220000ms , collection: control_collection slice: shard1 saw state=null with live_nodes=[]
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:816)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:781)
[junit4] 2> at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1178)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1079)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1014)
[junit4] 2> at org.apache.solr.core.ZkContainer.lambda$registerInZk$0(ZkContainer.java:181)
[junit4] 2> at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:208)
[junit4] 2> at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:886)
[junit4] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1047)
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:948)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:737)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:718)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:499)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1930023 ERROR (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkContainer :org.apache.solr.common.SolrException: Error getting leader from zk for shard shard1
[junit4] 2> at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1207)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1079)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1014)
[junit4] 2> at org.apache.solr.core.ZkContainer.lambda$registerInZk$0(ZkContainer.java:181)
[junit4] 2> at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:208)
[junit4] 2> at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:886)
[junit4] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1047)
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:948)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:737)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:718)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:499)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: org.apache.solr.common.SolrException: No registered leader was found after waiting for 1220000ms , collection: control_collection slice: shard1 saw state=null with live_nodes=[]
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:816)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:781)
[junit4] 2> at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1178)
[junit4] 2> ... 41 more
[junit4] 2>
[junit4] 2> 1930032 INFO (qtp10770406-11287) [n:127.0.0.1:37935_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=630169
[junit4] 2> 1930032 ERROR (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1930033 INFO (TEST-UnloadDistributedZkTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40619 40619
[junit4] 2> 1935069 INFO (Thread-2503) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40619 40619
[junit4] 2> 1935069 WARN (Thread-2503) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/clusterstate.json
[junit4] 2> 2 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/live_nodes
[junit4] 2> 2 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=UnloadDistributedZkTest -Dtests.method=test -Dtests.seed=1D36E40FDB869193 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=th-TH-u-nu-thai-x-lvariant-TH -Dtests.timezone=PRT -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 636s J0 | UnloadDistributedZkTest.test <<<
[junit4] > Throwable #1: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:37935
[junit4] > at __randomizedtesting.SeedInfo.seed([1D36E40FDB869193:9562DBD5757AFC6B]:0)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:883)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:314)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > Caused by: java.net.SocketTimeoutException: Read timed out
[junit4] > at java.net.SocketInputStream.socketRead0(Native Method)
[junit4] > at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
[junit4] > at java.net.SocketInputStream.read(SocketInputStream.java:171)
[junit4] > at java.net.SocketInputStream.read(SocketInputStream.java:141)
[junit4] > at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
[junit4] > at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
[junit4] > at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
[junit4] > at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
[junit4] > at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
[junit4] > at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
[junit4] > at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
[junit4] > at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
[junit4] > at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
[junit4] > at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
[junit4] > at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
[junit4] > at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
[junit4] > at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
[junit4] > at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
[junit4] > at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[junit4] > at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
[junit4] > at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
[junit4] > ... 43 more
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.UnloadDistributedZkTest_1D36E40FDB869193-001
[junit4] 2> NOTE: test params are: codec=FastCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST, chunkSize=3, maxDocsPerChunk=687, blockSize=9), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST, chunkSize=3, blockSize=9)), sim=RandomSimilarity(queryNorm=true): {}, locale=th-TH-u-nu-thai-x-lvariant-TH, timezone=PRT
[junit4] 2> NOTE: Linux 4.10.0-37-generic i386/Oracle Corporation 1.8.0_144 (32-bit)/cpus=8,threads=1,free=102891760,total=295460864
[junit4] 2> NOTE: All tests run in this JVM: [DistributedFacetPivotSmallTest, AutoScalingHandlerTest, MinimalSchemaTest, TestHalfAndHalfDocValues, TestRandomDVFaceting, HdfsBasicDistributedZk2Test, TestAuthorizationFramework, TestReplicationHandlerBackup, NodeLostTriggerTest, CleanupOldIndexTest, SoftAutoCommitTest, TestMergePolicyConfig, DirectUpdateHandlerOptimizeTest, SparseHLLTest, AtomicUpdatesTest, TestRecovery, AnalysisAfterCoreReloadTest, BlobRepositoryCloudTest, TestReplicationHandler, TestConfig, TestLuceneIndexBackCompat, TestNoOpRegenerator, TestFieldResource, BasicZkTest, CloudMLTQParserTest, TestComplexPhraseLeadingWildcard, TestMaxScoreQueryParser, FullHLLTest, SolrCoreTest, CursorPagingTest, TestNumericRangeQuery64, TestDFRSimilarityFactory, TestDistributedStatsComponentCardinality, TestRandomFaceting, TestRequestStatusCollectionAPI, CdcrBidirectionalTest, TestTlogReplica, DateMathParserTest, UninvertDocValuesMergePolicyTest, CacheHeaderTest, BlockDirectoryTest, OutOfBoxZkACLAndCredentialsProvidersTest, DistributedQueryComponentOptimizationTest, MoveReplicaTest, TestUtils, TestRawTransformer, TestCloudManagedSchema, TestConfigSetsAPI, TestInfoStreamLogging, SegmentsInfoRequestHandlerTest, SolrMetricManagerTest, RemoteQueryErrorTest, NumberUtilsTest, TestSchemaNameResource, BlockJoinFacetDistribTest, DistributedQueueTest, BadCopyFieldTest, TestUpdate, TestCodecSupport, DocumentBuilderTest, TestSchemaVersionResource, JvmMetricsTest, TlogReplayBufferedWhileIndexingTest, TestSolrIndexConfig, SolrInfoBeanTest, TestOrdValues, TestClusterProperties, TestNumericTerms32, TestBinaryField, TestHashQParserPlugin, TestSmileRequest, DistributedFacetPivotLargeTest, TestIBSimilarityFactory, TestSolrConfigHandlerConcurrent, ZkControllerTest, TestSort, TestTrieFacet, TestFieldCacheReopen, TestHighlightDedupGrouping, LeaderElectionIntegrationTest, SaslZkACLProviderTest, ExternalFileFieldSortTest, TestRTimerTree, TestLegacyTerms, UtilsToolTest, SolrXmlInZkTest, ResourceLoaderTest, SortSpecParsingTest, TestSolrCoreSnapshots, TestSolrConfigHandler, DistribCursorPagingTest, TestFreeTextSuggestions, TestBackupRepositoryFactory, TestSweetSpotSimilarityFactory, TestWordDelimiterFilterFactory, FileUtilsTest, SpatialFilterTest, TestJsonFacetRefinement, CdcrReplicationHandlerTest, DistributedDebugComponentTest, TestCSVLoader, SolrPluginUtilsTest, TestInPlaceUpdatesDistrib, HdfsSyncSliceTest, IndexSchemaRuntimeFieldTest, PeerSyncTest, DistributedQueryElevationComponentTest, JsonLoaderTest, UpdateParamsTest, DistributedMLTComponentTest, HdfsUnloadDistributedZkTest, TestScoreJoinQPNoScore, TestChildDocTransformer, TestRestManager, DateFieldTest, TestDistributedSearch, ConfigureRecoveryStrategyTest, SliceStateTest, ActionThrottleTest, TestCloudPseudoReturnFields, TestCrossCoreJoin, TestCustomStream, UnloadDistributedZkTest]
[junit4] Completed [376/748 (1!)] on J0 in 635.73s, 1 test, 1 error <<< FAILURES!
[...truncated 576 lines...]
[junit4] Suite: org.apache.solr.cloud.PeerSyncReplicationTest
[junit4] 2> 2613266 INFO (SUITE-PeerSyncReplicationTest-seed#[1D36E40FDB869193]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.PeerSyncReplicationTest_1D36E40FDB869193-001/init-core-data-001
[junit4] 2> 2613267 INFO (SUITE-PeerSyncReplicationTest-seed#[1D36E40FDB869193]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 2613269 INFO (SUITE-PeerSyncReplicationTest-seed#[1D36E40FDB869193]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 2613270 INFO (SUITE-PeerSyncReplicationTest-seed#[1D36E40FDB869193]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /
[junit4] 2> 2613273 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2613273 INFO (Thread-6811) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2613273 INFO (Thread-6811) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2613275 ERROR (Thread-6811) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2613373 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.ZkTestServer start zk server on port:39859
[junit4] 2> 2613385 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 2613387 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 2613388 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 2613389 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 2613390 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 2613391 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 2613392 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 2613393 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 2613394 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 2613395 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 2613396 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 2613400 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 2613483 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2613484 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6f2337{/,null,AVAILABLE}
[junit4] 2> 2613485 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@9d48eb{SSL,[ssl, http/1.1]}{127.0.0.1:45929}
[junit4] 2> 2613485 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.e.j.s.Server Started @2615034ms
[junit4] 2> 2613485 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.PeerSyncReplicationTest_1D36E40FDB869193-001/tempDir-001/control/data, replicaType=NRT, hostContext=/, hostPort=45929, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.PeerSyncReplicationTest_1D36E40FDB869193-001/control-001/cores}
[junit4] 2> 2613485 ERROR (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2613485 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.2.0
[junit4] 2> 2613486 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2613486 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2613486 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-10T13:08:45.288Z
[junit4] 2> 2613489 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 2613489 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.PeerSyncReplicationTest_1D36E40FDB869193-001/control-001/solr.xml
[junit4] 2> 2613491 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2613491 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2613492 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@71c67a, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2613494 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:39859/solr
[junit4] 2> 2613535 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [n:127.0.0.1:45929_ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2613535 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [n:127.0.0.1:45929_ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:45929_
[junit4] 2> 2613536 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [n:127.0.0.1:45929_ ] o.a.s.c.Overseer Overseer (id=98980287288180741-127.0.0.1:45929_-n_0000000000) starting
[junit4] 2> 2613539 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [n:127.0.0.1:45929_ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45929_
[junit4] 2> 2613539 INFO (zkCallback-1809-thread-1-processing-n:127.0.0.1:45929_) [n:127.0.0.1:45929_ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2613612 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [n:127.0.0.1:45929_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@71c67a
[junit4] 2> 2613619 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [n:127.0.0.1:45929_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@71c67a
[junit4] 2> 2613619 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [n:127.0.0.1:45929_ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@71c67a
[junit4] 2> 2613620 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [n:127.0.0.1:45929_ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.PeerSyncReplicationTest_1D36E40FDB869193-001/control-001/cores
[junit4] 2> 2613633 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2613633 INFO (TEST-PeerSyncReplicationTest.test-seed#[1D36E40FDB869193]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:39859/solr ready
[junit4] 2> 2613654 INFO (qtp20358503-21926) [n:127.0.0.1:45929_ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:45929_&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2613655 INFO (OverseerThreadFactory-5130-thread-1-processing-n:127.0.0.1:45929_) [n:127.0.0.1:45929_ ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 2613763 INFO (qtp20358503-21920) [n:127.0.0.1:45929_ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2613764 INFO (qtp20358503-21920) [n:127.0.0.1:45929_ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2613869 INFO (zkCallback-1809-thread-1-processing-n:127.0.0.1:45929_) [n:127.0.0.1:45929_ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2614800 INFO (qtp20358503-21920) [n:127.0.0.1:45929_ c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Us
[...truncated too long message...]
nit4] 2> 70386 INFO (coreCloseExecutor-463-thread-6) [n:127.0.0.1:36949_solr c:2nd_collection s:shard2 r:core_node4 x:2nd_collection_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.2nd_collection.shard2.replica_n2, tag=2440585
[junit4] 2> 70386 INFO (coreCloseExecutor-463-thread-6) [n:127.0.0.1:36949_solr c:2nd_collection s:shard2 r:core_node4 x:2nd_collection_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1dad64c: rootName = solr_36949, domain = solr.core.2nd_collection.shard2.replica_n2, service url = null, agent id = null] for registry solr.core.2nd_collection.shard2.replica_n2 / com.codahale.metrics.MetricRegistry@1818ffc
[junit4] 2> 70391 INFO (jetty-closer-30-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@15f60d0{/solr,null,UNAVAILABLE}
[junit4] 2> 70421 INFO (coreCloseExecutor-462-thread-6) [n:127.0.0.1:40675_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.stale_state_test_col.shard1.replica_n3, tag=16339540
[junit4] 2> 70421 INFO (coreCloseExecutor-463-thread-7) [n:127.0.0.1:36949_solr c:foo s:shard1 r:core_node6 x:foo_shard1_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.foo.shard1.replica_n4, tag=8833226
[junit4] 2> 70421 INFO (coreCloseExecutor-462-thread-6) [n:127.0.0.1:40675_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@8acd19: rootName = solr_40675, domain = solr.core.stale_state_test_col.shard1.replica_n3, service url = null, agent id = null] for registry solr.core.stale_state_test_col.shard1.replica_n3 / com.codahale.metrics.MetricRegistry@1632dd8
[junit4] 2> 70421 INFO (coreCloseExecutor-463-thread-7) [n:127.0.0.1:36949_solr c:foo s:shard1 r:core_node6 x:foo_shard1_replica_n4] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1674b50: rootName = solr_36949, domain = solr.core.foo.shard1.replica_n4, service url = null, agent id = null] for registry solr.core.foo.shard1.replica_n4 / com.codahale.metrics.MetricRegistry@e9b1be
[junit4] 2> 70439 INFO (coreCloseExecutor-462-thread-1) [n:127.0.0.1:40675_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard2.leader, tag=29730609
[junit4] 2> 70439 INFO (coreCloseExecutor-462-thread-2) [n:127.0.0.1:40675_solr c:multicollection2 s:shard2 r:core_node4 x:multicollection2_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.multicollection2.shard2.leader, tag=14363516
[junit4] 2> 70439 INFO (coreCloseExecutor-462-thread-3) [n:127.0.0.1:40675_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n16] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard3.leader, tag=5255919
[junit4] 2> 70439 INFO (coreCloseExecutor-462-thread-4) [n:127.0.0.1:40675_solr c:localShardsTestColl s:shard2 r:core_node13 x:localShardsTestColl_shard2_replica_n10] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard2.leader, tag=2177144
[junit4] 2> 70440 INFO (coreCloseExecutor-462-thread-6) [n:127.0.0.1:40675_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.stale_state_test_col.shard1.leader, tag=16339540
[junit4] 2> 70440 INFO (coreCloseExecutor-463-thread-1) [n:127.0.0.1:36949_solr c:overwrite s:shard1 r:core_node2 x:overwrite_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.overwrite.shard1.leader, tag=18650975
[junit4] 2> 70440 INFO (coreCloseExecutor-463-thread-2) [n:127.0.0.1:36949_solr c:multicollection1 s:shard2 r:core_node4 x:multicollection1_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.multicollection1.shard2.leader, tag=4094097
[junit4] 2> 70440 INFO (coreCloseExecutor-463-thread-3) [n:127.0.0.1:36949_solr c:localShardsTestColl s:shard1 r:core_node2 x:localShardsTestColl_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard1.leader, tag=32220360
[junit4] 2> 70440 INFO (coreCloseExecutor-463-thread-7) [n:127.0.0.1:36949_solr c:foo s:shard1 r:core_node6 x:foo_shard1_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.foo.shard1.leader, tag=8833226
[junit4] 2> 70440 INFO (coreCloseExecutor-462-thread-7) [n:127.0.0.1:40675_solr c:foo s:shard1 r:core_node5 x:foo_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.foo.shard1.leader, tag=9479622
[junit4] 2> 70441 INFO (coreCloseExecutor-463-thread-6) [n:127.0.0.1:36949_solr c:2nd_collection s:shard2 r:core_node4 x:2nd_collection_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.2nd_collection.shard2.leader, tag=2440585
[junit4] 2> 70455 INFO (coreCloseExecutor-462-thread-5) [n:127.0.0.1:40675_solr c:localShardsTestColl s:shard1 r:core_node7 x:localShardsTestColl_shard1_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard1.leader, tag=680497
[junit4] 2> 70534 INFO (jetty-closer-30-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1f228ae{/solr,null,UNAVAILABLE}
[junit4] 2> 70538 INFO (jetty-closer-30-thread-2) [ ] o.a.s.c.Overseer Overseer (id=98980350546739209-127.0.0.1:40675_solr-n_0000000000) closing
[junit4] 2> 70538 INFO (OverseerStateUpdate-98980350546739209-127.0.0.1:40675_solr-n_0000000000) [n:127.0.0.1:40675_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:40675_solr
[junit4] 2> 70569 WARN (OverseerAutoScalingTriggerThread-98980350546739209-127.0.0.1:40675_solr-n_0000000000) [n:127.0.0.1:40675_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 70585 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15fa61aba150009, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 70585 INFO (jetty-closer-30-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@15e8a2d{/solr,null,UNAVAILABLE}
[junit4] 2> 70587 ERROR (SUITE-CloudSolrClientTest-seed#[52C158DA1A9AA452]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 70588 INFO (SUITE-CloudSolrClientTest-seed#[52C158DA1A9AA452]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:43625 43625
[junit4] 2> 75871 INFO (Thread-65) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:43625 43625
[junit4] 2> 75872 WARN (Thread-65) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/aliases.json
[junit4] 2> 3 /solr/security.json
[junit4] 2> 3 /solr/configs/conf
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 27 /solr/collections/foo/state.json
[junit4] 2> 27 /solr/collections/localShardsTestColl/state.json
[junit4] 2> 16 /solr/collections/stale_state_test_col/state.json
[junit4] 2> 14 /solr/collections/collection1/state.json
[junit4] 2> 14 /solr/collections/multicollection2/state.json
[junit4] 2> 12 /solr/collections/2nd_collection/state.json
[junit4] 2> 12 /solr/collections/multicollection1/state.json
[junit4] 2> 10 /solr/collections/nemesis/state.json
[junit4] 2> 8 /solr/clusterprops.json
[junit4] 2> 8 /solr/clusterstate.json
[junit4] 2> 6 /solr/collections/overwrite/state.json
[junit4] 2> 2 /solr/collections/localShardsTestColl/leader_elect/shard1/election/98980350546739208-core_node2-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/live_nodes
[junit4] 2> 8 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.impl.CloudSolrClientTest_52C158DA1A9AA452-001
[junit4] 2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=true): {}, locale=es-CU, timezone=America/Argentina/Tucuman
[junit4] 2> NOTE: Linux 4.10.0-37-generic i386/Oracle Corporation 1.8.0_144 (32-bit)/cpus=8,threads=1,free=90435392,total=217911296
[junit4] 2> NOTE: All tests run in this JVM: [HttpSolrClientBuilderTest, TestSuggesterResponse, ModifiableSolrParamsTest, ArrayEvaluatorTest, TestSolrJErrorHandling, TestValidatingJsonMap, SolrExampleBinaryTest, JdbcDriverTest, TestCoreAdmin, AddEvaluatorTest, CloudSolrClientTest]
[junit4] Completed [84/146 (1!)] on J2 in 41.80s, 16 tests, 1 error <<< FAILURES!
[...truncated 42364 lines...]
[JENKINS] Lucene-Solr-7.x-Linux (64bit/jdk1.8.0_144) - Build # 786
- Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/786/
Java: 64bit/jdk1.8.0_144 -XX:+UseCompressedOops -XX:+UseSerialGC
3 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
Error Message:
2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation: 1) Thread[id=17143, name=jetty-launcher-3221-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105) at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41) at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244) at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44) at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61) at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505) 2) Thread[id=17131, name=jetty-launcher-3221-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037) at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277) at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323) at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105) at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279) at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41) at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244) at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44) at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61) at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67) at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530) at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 2 threads leaked from SUITE scope at org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation:
1) Thread[id=17143, name=jetty-launcher-3221-thread-1-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
2) Thread[id=17131, name=jetty-launcher-3221-thread-2-EventThread, state=TIMED_WAITING, group=TGRP-TestSolrCloudWithSecureImpersonation]
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
at org.apache.curator.CuratorZookeeperClient.internalBlockUntilConnectedOrTimedOut(CuratorZookeeperClient.java:323)
at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:105)
at org.apache.curator.framework.imps.GetDataBuilderImpl.pathInForeground(GetDataBuilderImpl.java:288)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:279)
at org.apache.curator.framework.imps.GetDataBuilderImpl.forPath(GetDataBuilderImpl.java:41)
at org.apache.curator.framework.recipes.shared.SharedValue.readValue(SharedValue.java:244)
at org.apache.curator.framework.recipes.shared.SharedValue.access$100(SharedValue.java:44)
at org.apache.curator.framework.recipes.shared.SharedValue$1.process(SharedValue.java:61)
at org.apache.curator.framework.imps.NamespaceWatcher.process(NamespaceWatcher.java:67)
at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:530)
at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:505)
at __randomizedtesting.SeedInfo.seed([D0BA77A24A54D903]:0)
FAILED: org.apache.solr.cloud.CollectionsAPISolrJTest.testAddAndDeleteReplicaProp
Error Message:
Expecting property 'preferredleader' to appear on replica core_node5 null Live Nodes: [127.0.0.1:40605_solr, 127.0.0.1:41525_solr, 127.0.0.1:42337_solr, 127.0.0.1:45179_solr] Last available state: DocCollection(replicaProperties//collections/replicaProperties/state.json/5)={ "pullReplicas":"0", "replicationFactor":"2", "shards":{ "shard1":{ "range":"80000000-ffffffff", "state":"active", "replicas":{ "core_node3":{ "core":"replicaProperties_shard1_replica_n1", "base_url":"https://127.0.0.1:45179/solr", "node_name":"127.0.0.1:45179_solr", "state":"active", "type":"NRT"}, "core_node5":{ "core":"replicaProperties_shard1_replica_n2", "base_url":"https://127.0.0.1:41525/solr", "node_name":"127.0.0.1:41525_solr", "state":"active", "type":"NRT", "leader":"true"}}}, "shard2":{ "range":"0-7fffffff", "state":"active", "replicas":{ "core_node7":{ "core":"replicaProperties_shard2_replica_n4", "base_url":"https://127.0.0.1:42337/solr", "node_name":"127.0.0.1:42337_solr", "state":"active", "type":"NRT"}, "core_node8":{ "core":"replicaProperties_shard2_replica_n6", "base_url":"https://127.0.0.1:40605/solr", "node_name":"127.0.0.1:40605_solr", "state":"active", "type":"NRT", "leader":"true"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"1", "autoAddReplicas":"false", "nrtReplicas":"2", "tlogReplicas":"0"}
Stack Trace:
java.lang.AssertionError: Expecting property 'preferredleader' to appear on replica core_node5
null
Live Nodes: [127.0.0.1:40605_solr, 127.0.0.1:41525_solr, 127.0.0.1:42337_solr, 127.0.0.1:45179_solr]
Last available state: DocCollection(replicaProperties//collections/replicaProperties/state.json/5)={
"pullReplicas":"0",
"replicationFactor":"2",
"shards":{
"shard1":{
"range":"80000000-ffffffff",
"state":"active",
"replicas":{
"core_node3":{
"core":"replicaProperties_shard1_replica_n1",
"base_url":"https://127.0.0.1:45179/solr",
"node_name":"127.0.0.1:45179_solr",
"state":"active",
"type":"NRT"},
"core_node5":{
"core":"replicaProperties_shard1_replica_n2",
"base_url":"https://127.0.0.1:41525/solr",
"node_name":"127.0.0.1:41525_solr",
"state":"active",
"type":"NRT",
"leader":"true"}}},
"shard2":{
"range":"0-7fffffff",
"state":"active",
"replicas":{
"core_node7":{
"core":"replicaProperties_shard2_replica_n4",
"base_url":"https://127.0.0.1:42337/solr",
"node_name":"127.0.0.1:42337_solr",
"state":"active",
"type":"NRT"},
"core_node8":{
"core":"replicaProperties_shard2_replica_n6",
"base_url":"https://127.0.0.1:40605/solr",
"node_name":"127.0.0.1:40605_solr",
"state":"active",
"type":"NRT",
"leader":"true"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"1",
"autoAddReplicas":"false",
"nrtReplicas":"2",
"tlogReplicas":"0"}
at __randomizedtesting.SeedInfo.seed([D0BA77A24A54D903:1461C84AF17B2A6F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:269)
at org.apache.solr.cloud.CollectionsAPISolrJTest.testAddAndDeleteReplicaProp(CollectionsAPISolrJTest.java:371)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.ShardSplitTest.test
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:33175/i/x
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:33175/i/x
at __randomizedtesting.SeedInfo.seed([D0BA77A24A54D903:58EE4878E4A8B4FB]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:883)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:314)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
... 43 more
Build Log:
[...truncated 12386 lines...]
[junit4] Suite: org.apache.solr.cloud.TestSolrCloudWithSecureImpersonation
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_D0BA77A24A54D903-001/init-core-data-001
[junit4] 2> 742551 WARN (SUITE-TestSolrCloudWithSecureImpersonation-seed#[D0BA77A24A54D903]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=19 numCloses=19
[junit4] 2> 742551 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[D0BA77A24A54D903]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 742552 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[D0BA77A24A54D903]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 742552 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[D0BA77A24A54D903]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 742564 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[D0BA77A24A54D903]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_D0BA77A24A54D903-001/tempDir-001
[junit4] 2> 742564 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[D0BA77A24A54D903]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 742564 INFO (Thread-3574) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 742564 INFO (Thread-3574) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 742566 ERROR (Thread-3574) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 742664 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[D0BA77A24A54D903]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:36937
[junit4] 2> 742668 INFO (jetty-launcher-3221-thread-2) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 742668 INFO (jetty-launcher-3221-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 742669 INFO (jetty-launcher-3221-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1a311aa9{/solr,null,AVAILABLE}
[junit4] 2> 742669 INFO (jetty-launcher-3221-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@36b3167e{/solr,null,AVAILABLE}
[junit4] 2> 742670 INFO (jetty-launcher-3221-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@366d6954{HTTP/1.1,[http/1.1]}{127.0.0.1:41609}
[junit4] 2> 742670 INFO (jetty-launcher-3221-thread-1) [ ] o.e.j.s.Server Started @745076ms
[junit4] 2> 742670 INFO (jetty-launcher-3221-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=41609}
[junit4] 2> 742670 INFO (jetty-launcher-3221-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@128fe973{HTTP/1.1,[http/1.1]}{127.0.0.1:44841}
[junit4] 2> 742670 INFO (jetty-launcher-3221-thread-2) [ ] o.e.j.s.Server Started @745076ms
[junit4] 2> 742670 INFO (jetty-launcher-3221-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44841}
[junit4] 2> 742670 ERROR (jetty-launcher-3221-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 742670 INFO (jetty-launcher-3221-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.2.0
[junit4] 2> 742670 ERROR (jetty-launcher-3221-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 742670 INFO (jetty-launcher-3221-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 742670 INFO (jetty-launcher-3221-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.2.0
[junit4] 2> 742670 INFO (jetty-launcher-3221-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 742670 INFO (jetty-launcher-3221-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 742670 INFO (jetty-launcher-3221-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-10T09:27:04.569Z
[junit4] 2> 742670 INFO (jetty-launcher-3221-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 742670 INFO (jetty-launcher-3221-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-10T09:27:04.569Z
[junit4] 2> 742671 INFO (jetty-launcher-3221-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 742671 INFO (jetty-launcher-3221-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 742676 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15fa5410c100002, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 742679 INFO (jetty-launcher-3221-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36937/solr
[junit4] 2> 742679 INFO (jetty-launcher-3221-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:36937/solr
[junit4] 2> 742715 INFO (jetty-launcher-3221-thread-1) [n:127.0.0.1:41609_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 742715 INFO (jetty-launcher-3221-thread-2) [n:127.0.0.1:44841_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 742715 INFO (jetty-launcher-3221-thread-1) [n:127.0.0.1:41609_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:41609_solr
[junit4] 2> 742715 INFO (jetty-launcher-3221-thread-2) [n:127.0.0.1:44841_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44841_solr
[junit4] 2> 742716 INFO (jetty-launcher-3221-thread-1) [n:127.0.0.1:41609_solr ] o.a.s.c.Overseer Overseer (id=98979415619272709-127.0.0.1:41609_solr-n_0000000000) starting
[junit4] 2> 742719 INFO (zkCallback-3232-thread-1-processing-n:127.0.0.1:41609_solr) [n:127.0.0.1:41609_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 742719 INFO (zkCallback-3233-thread-1-processing-n:127.0.0.1:44841_solr) [n:127.0.0.1:44841_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 742722 INFO (jetty-launcher-3221-thread-1) [n:127.0.0.1:41609_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:41609_solr
[junit4] 2> 742723 INFO (zkCallback-3232-thread-1-processing-n:127.0.0.1:41609_solr) [n:127.0.0.1:41609_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 742723 INFO (zkCallback-3233-thread-1-processing-n:127.0.0.1:44841_solr) [n:127.0.0.1:44841_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 742749 INFO (jetty-launcher-3221-thread-2) [n:127.0.0.1:44841_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
[junit4] 2> 742752 WARN (jetty-launcher-3221-thread-2) [n:127.0.0.1:44841_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 742753 INFO (jetty-launcher-3221-thread-2) [n:127.0.0.1:44841_solr ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=localhost.localdomain,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=jenkins, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
[junit4] 2> 742756 INFO (jetty-launcher-3221-thread-2) [n:127.0.0.1:44841_solr ] o.a.c.f.i.CuratorFrameworkImpl Starting
[junit4] 2> 742760 INFO (jetty-launcher-3221-thread-2-EventThread) [n:127.0.0.1:44841_solr ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
[junit4] 2> 742767 WARN (jetty-launcher-3221-thread-2) [n:127.0.0.1:44841_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 742795 INFO (jetty-launcher-3221-thread-2) [n:127.0.0.1:44841_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44841.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6392c839
[junit4] 2> 742801 INFO (jetty-launcher-3221-thread-2) [n:127.0.0.1:44841_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44841.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6392c839
[junit4] 2> 742801 INFO (jetty-launcher-3221-thread-2) [n:127.0.0.1:44841_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44841.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6392c839
[junit4] 2> 742802 INFO (jetty-launcher-3221-thread-2) [n:127.0.0.1:44841_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_D0BA77A24A54D903-001/tempDir-001/node2/.
[junit4] 2> 742804 INFO (jetty-launcher-3221-thread-1) [n:127.0.0.1:41609_solr ] o.a.s.c.CoreContainer Initializing authentication plugin: org.apache.solr.security.HttpParamDelegationTokenPlugin
[junit4] 2> 742804 WARN (jetty-launcher-3221-thread-1) [n:127.0.0.1:41609_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 742805 INFO (jetty-launcher-3221-thread-1) [n:127.0.0.1:41609_solr ] o.a.s.s.KerberosPlugin Params: {solr.kerberos.impersonator.user.noGroups.hosts=*, solr.kerberos.impersonator.user.bogusGroup.groups=__some_bogus_group, token.valid=30, cookie.domain=127.0.0.1, solr.kerberos.impersonator.user.anyHostAnyUser.groups=*, solr.kerberos.impersonator.user.bogusGroup.hosts=*, solr.kerberos.impersonator.user.anyHostAnyUser.hosts=*, token.validity=36000, solr.kerberos.impersonator.user.noHosts.groups=*, type=kerberos, solr.kerberos.impersonator.user.localHostAnyGroup.groups=*, delegation-token.token-kind=solr-dt, solr.kerberos.impersonator.user.wrongHost.hosts=1.1.1.1.1.1, cookie.path=/, zk-dt-secret-manager.znodeWorkingPath=solr/security/zkdtsm, solr.kerberos.impersonator.user.localHostAnyGroup.hosts=localhost.localdomain,localhost,127.0.0.1, signer.secret.provider.zookeeper.path=/token, solr.kerberos.impersonator.user.anyHostUsersGroup.groups=jenkins, zk-dt-secret-manager.enable=true, solr.kerberos.impersonator.user.wrongHost.groups=*, kerberos.name.rules=DEFAULT, signer.secret.provider=zookeeper, solr.kerberos.impersonator.user.anyHostUsersGroup.hosts=*}
[junit4] 2> 742805 INFO (jetty-launcher-3221-thread-1) [n:127.0.0.1:41609_solr ] o.a.c.f.i.CuratorFrameworkImpl Starting
[junit4] 2> 742806 INFO (jetty-launcher-3221-thread-1-EventThread) [n:127.0.0.1:41609_solr ] o.a.c.f.s.ConnectionStateManager State change: CONNECTED
[junit4] 2> 742810 WARN (jetty-launcher-3221-thread-1) [n:127.0.0.1:41609_solr ] o.a.s.c.s.i.Krb5HttpClientBuilder org.apache.solr.client.solrj.impl.Krb5HttpClientBuilder is configured without specifying system property 'java.security.auth.login.config'
[junit4] 2> 742849 INFO (jetty-launcher-3221-thread-1) [n:127.0.0.1:41609_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41609.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6392c839
[junit4] 2> 742854 INFO (jetty-launcher-3221-thread-1) [n:127.0.0.1:41609_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41609.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6392c839
[junit4] 2> 742854 INFO (jetty-launcher-3221-thread-1) [n:127.0.0.1:41609_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_41609.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6392c839
[junit4] 2> 742855 INFO (jetty-launcher-3221-thread-1) [n:127.0.0.1:41609_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_D0BA77A24A54D903-001/tempDir-001/node1/.
[junit4] 2> 742868 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15fa5410c100009, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 742879 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testForwarding
[junit4] 2> 743012 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[D0BA77A24A54D903]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 743012 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[D0BA77A24A54D903]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:36937/solr ready
[junit4] 2> 743020 INFO (qtp1912366258-17093) [n:127.0.0.1:44841_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=1&collection.configName=conf1&maxShardsPerNode=1&name=forwardingCollection&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 743022 INFO (OverseerThreadFactory-3889-thread-1-processing-n:127.0.0.1:41609_solr) [n:127.0.0.1:41609_solr ] o.a.s.c.CreateCollectionCmd Create collection forwardingCollection
[junit4] 2> 743125 INFO (OverseerStateUpdate-98979415619272709-127.0.0.1:41609_solr-n_0000000000) [n:127.0.0.1:41609_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"forwardingCollection",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"forwardingCollection_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:44841/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 743327 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=forwardingCollection_shard1_replica_n1&action=CREATE&numShards=1&collection=forwardingCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 743327 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 743429 INFO (zkCallback-3233-thread-1-processing-n:127.0.0.1:44841_solr) [n:127.0.0.1:44841_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 743430 INFO (zkCallback-3233-thread-2-processing-n:127.0.0.1:44841_solr) [n:127.0.0.1:44841_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 744339 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.2.0
[junit4] 2> 744352 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.IndexSchema [forwardingCollection_shard1_replica_n1] Schema name=test
[junit4] 2> 744426 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 744433 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'forwardingCollection_shard1_replica_n1' using configuration from collection forwardingCollection, trusted=true
[junit4] 2> 744433 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44841.solr.core.forwardingCollection.shard1.replica_n1' (registry 'solr.core.forwardingCollection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@6392c839
[junit4] 2> 744434 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 744434 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [[forwardingCollection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_D0BA77A24A54D903-001/tempDir-001/node2/forwardingCollection_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J2/temp/solr.cloud.TestSolrCloudWithSecureImpersonation_D0BA77A24A54D903-001/tempDir-001/node2/./forwardingCollection_shard1_replica_n1/data/]
[junit4] 2> 744435 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=15, maxMergeAtOnceExplicit=28, maxMergedSegmentMB=10.6748046875, floorSegmentMB=2.1513671875, forceMergeDeletesPctAllowed=25.601487941026765, segmentsPerTier=10.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.16916501092871056
[junit4] 2> 744471 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 744471 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 744485 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 744485 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 744486 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=48, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7439564935952828]
[junit4] 2> 744486 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@58012bd0[forwardingCollection_shard1_replica_n1] main]
[junit4] 2> 744487 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Initializing spell checkers
[junit4] 2> 744506 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.DirectSolrSpellChecker init: {name=direct,classname=DirectSolrSpellChecker,field=lowerfilt,minQueryLength=3}
[junit4] 2> 744570 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 744570 INFO (searcherExecutor-3894-thread-1-processing-n:127.0.0.1:44841_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: default
[junit4] 2> 744570 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1583670652010758144
[junit4] 2> 744570 INFO (searcherExecutor-3894-thread-1-processing-n:127.0.0.1:44841_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: direct
[junit4] 2> 744570 INFO (searcherExecutor-3894-thread-1-processing-n:127.0.0.1:44841_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: wordbreak
[junit4] 2> 744570 INFO (searcherExecutor-3894-thread-1-processing-n:127.0.0.1:44841_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: multipleFields
[junit4] 2> 744571 INFO (searcherExecutor-3894-thread-1-processing-n:127.0.0.1:44841_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: jarowinkler
[junit4] 2> 744571 INFO (searcherExecutor-3894-thread-1-processing-n:127.0.0.1:44841_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: external
[junit4] 2> 744571 INFO (searcherExecutor-3894-thread-1-processing-n:127.0.0.1:44841_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: freq
[junit4] 2> 744571 INFO (searcherExecutor-3894-thread-1-processing-n:127.0.0.1:44841_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: fqcn
[junit4] 2> 744571 INFO (searcherExecutor-3894-thread-1-processing-n:127.0.0.1:44841_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.h.c.SpellCheckComponent Loading spell index for spellchecker: perDict
[junit4] 2> 744572 INFO (searcherExecutor-3894-thread-1-processing-n:127.0.0.1:44841_solr x:forwardingCollection_shard1_replica_n1 s:shard1 c:forwardingCollection r:core_node2) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [forwardingCollection_shard1_replica_n1] Registered new searcher Searcher@58012bd0[forwardingCollection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 744573 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 744573 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 744573 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:44841/solr/forwardingCollection_shard1_replica_n1/
[junit4] 2> 744573 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 744573 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:44841/solr/forwardingCollection_shard1_replica_n1/ has no replicas
[junit4] 2> 744573 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 744575 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:44841/solr/forwardingCollection_shard1_replica_n1/ shard1
[junit4] 2> 744676 INFO (zkCallback-3233-thread-2-processing-n:127.0.0.1:44841_solr) [n:127.0.0.1:44841_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 744676 INFO (zkCallback-3233-thread-1-processing-n:127.0.0.1:44841_solr) [n:127.0.0.1:44841_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 744725 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 744726 INFO (qtp1912366258-17097) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&newCollection=true&name=forwardingCollection_shard1_replica_n1&action=CREATE&numShards=1&collection=forwardingCollection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1399
[junit4] 2> 744727 WARN (httpShardExecutor-3881-thread-1-processing-n:127.0.0.1:41609_solr [http:////127.0.0.1:44841//solr] http:////127.0.0.1:44841//solr) [n:127.0.0.1:41609_solr ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1510342025225&s=96qHAPlVOd57kNmQJqELxI3kKpI="; Path=/; Domain=127.0.0.1; Expires= , 10-11-2017 19:27:05 GMT; HttpOnly". Invalid 'expires' attribute: , 10-11-2017 19:27:05 GMT
[junit4] 2> 744729 INFO (qtp1912366258-17093) [n:127.0.0.1:44841_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 744827 INFO (zkCallback-3233-thread-1-processing-n:127.0.0.1:44841_solr) [n:127.0.0.1:44841_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 744827 INFO (zkCallback-3233-thread-2-processing-n:127.0.0.1:44841_solr) [n:127.0.0.1:44841_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [2])
[junit4] 2> 745022 INFO (OverseerCollectionConfigSetProcessor-98979415619272709-127.0.0.1:41609_solr-n_0000000000) [n:127.0.0.1:41609_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 745729 INFO (qtp1912366258-17093) [n:127.0.0.1:44841_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&replicationFactor=1&collection.configName=conf1&maxShardsPerNode=1&name=forwardingCollection&nrtReplicas=1&action=CREATE&numShards=1&tlogReplicas=0&wt=javabin&version=2} status=0 QTime=2709
[junit4] 2> 745729 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[D0BA77A24A54D903]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1510342024914&s=ZDNgb7UHTUxYiaQBsDcaDZAMkxo="; Path=/; Domain=127.0.0.1; Expires= , 10-11-2017 19:27:04 GMT; HttpOnly". Invalid 'expires' attribute: , 10-11-2017 19:27:04 GMT
[junit4] 2> 745729 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[D0BA77A24A54D903]) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: forwardingCollection failOnTimeout:true timeout (sec):100
[junit4] 2> 745730 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[D0BA77A24A54D903]) [ ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: forwardingCollection
[junit4] 2> 745732 INFO (qtp1912366258-17085) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.S.Request [forwardingCollection_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 745732 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[D0BA77A24A54D903]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1510342027630&s=3+UJ4IYNuxw6dZ0LYrYNzw3mz9M="; Path=/; Domain=127.0.0.1; Expires= , 10-11-2017 19:27:07 GMT; HttpOnly". Invalid 'expires' attribute: , 10-11-2017 19:27:07 GMT
[junit4] 2> 745734 INFO (qtp267470422-17094) [n:127.0.0.1:41609_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 745739 INFO (qtp1912366258-17085) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.S.Request [forwardingCollection_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 745739 WARN (qtp267470422-17094) [n:127.0.0.1:41609_solr ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1510342027638&s=1cKXztzetb9bpwTy3Oe2Cv8iFyc="; Path=/; Domain=127.0.0.1; Expires= , 10-11-2017 19:27:07 GMT; HttpOnly". Invalid 'expires' attribute: , 10-11-2017 19:27:07 GMT
[junit4] 2> 745740 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[D0BA77A24A54D903]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=user&p=user&t=test&e=1510342027633&s=mqRtxcSWolsBa5uC9JcseffkiOQ="; Path=/; Domain=127.0.0.1; Expires= , 10-11-2017 19:27:07 GMT; HttpOnly". Invalid 'expires' attribute: , 10-11-2017 19:27:07 GMT
[junit4] 2> 745740 WARN (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[D0BA77A24A54D903]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=$&p=$&t=test&e=1510342027638&s=1cKXztzetb9bpwTy3Oe2Cv8iFyc="; Path=/; Domain=127.0.0.1; Expires= , 10-11-2017 19:27:07 GMT; HttpOnly". Invalid 'expires' attribute: , 10-11-2017 19:27:07 GMT
[junit4] 2> 745740 INFO (TEST-TestSolrCloudWithSecureImpersonation.testForwarding-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testForwarding
[junit4] 2> 745754 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidProxyUser
[junit4] 2> 745756 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[D0BA77A24A54D903]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bar&p=bar&t=test&e=1510342027653&s=PvfSFSWZRAYHIpNu/CYMd9UTa4Q="; Path=/; Domain=127.0.0.1; Expires= , 10-11-2017 19:27:07 GMT; HttpOnly". Invalid 'expires' attribute: , 10-11-2017 19:27:07 GMT
[junit4] 2> 745757 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidProxyUser-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidProxyUser
[junit4] 2> 745769 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidGroup
[junit4] 2> 745775 WARN (qtp1912366258-17089) [ ] o.a.h.s.ShellBasedUnixGroupsMapping got exception trying to get groups for user bar: id: ???bar???: no such user
[junit4] 2> id: ???bar???: no such user
[junit4] 2>
[junit4] 2> 745776 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[D0BA77A24A54D903]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=bogusGroup&p=bogusGroup&t=test&e=1510342027668&s=2YlPKTr9fRZV4RGe9sIDHQx/mWY="; Path=/; Domain=127.0.0.1; Expires= , 10-11-2017 19:27:07 GMT; HttpOnly". Invalid 'expires' attribute: , 10-11-2017 19:27:07 GMT
[junit4] 2> 745777 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidGroup-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidGroup
[junit4] 2> 745788 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigHosts
[junit4] 2> 745789 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[D0BA77A24A54D903]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=noHosts&p=noHosts&t=test&e=1510342027687&s=XzCr6F6Kyw8/5yvsXa98aLgHvb8="; Path=/; Domain=127.0.0.1; Expires= , 10-11-2017 19:27:07 GMT; HttpOnly". Invalid 'expires' attribute: , 10-11-2017 19:27:07 GMT
[junit4] 2> 745789 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigHosts-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigHosts
[junit4] 2> 745801 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyInvalidRemote
[junit4] 2> 745822 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[D0BA77A24A54D903]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1510342027701&s=/q9GzTus15EbieI1h6yDX0UvUwk="; Path=/; Domain=127.0.0.1; Expires= , 10-11-2017 19:27:07 GMT; HttpOnly". Invalid 'expires' attribute: , 10-11-2017 19:27:07 GMT
[junit4] 2> 745822 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyInvalidRemote-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyInvalidRemote
[junit4] 2> 745833 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateGroup
[junit4] 2> 745839 INFO (qtp1912366258-17093) [n:127.0.0.1:44841_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=jenkins&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 745839 INFO (qtp1912366258-17093) [n:127.0.0.1:44841_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=jenkins&action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 745840 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[D0BA77A24A54D903]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=anyHostUsersGroup&p=anyHostUsersGroup&t=test&e=1510342027732&s=wwIOQFeKz5XArV/4FYZUe4FIvIY="; Path=/; Domain=127.0.0.1; Expires= , 10-11-2017 19:27:07 GMT; HttpOnly". Invalid 'expires' attribute: , 10-11-2017 19:27:07 GMT
[junit4] 2> 745840 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateGroup-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateGroup
[junit4] 2> 745853 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateAnyHostAnyUser
[junit4] 2> 745853 INFO (qtp1912366258-17089) [n:127.0.0.1:44841_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 745853 INFO (qtp1912366258-17089) [n:127.0.0.1:44841_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 745853 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[D0BA77A24A54D903]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=anyHostAnyUser&p=anyHostAnyUser&t=test&e=1510342027752&s=rN8cqACyR8SDXhkNe4CQU5WVguM="; Path=/; Domain=127.0.0.1; Expires= , 10-11-2017 19:27:07 GMT; HttpOnly". Invalid 'expires' attribute: , 10-11-2017 19:27:07 GMT
[junit4] 2> 745854 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateAnyHostAnyUser-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateAnyHostAnyUser
[junit4] 2> 745864 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyValidateHost
[junit4] 2> 745865 INFO (qtp1912366258-17085) [n:127.0.0.1:44841_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :list with params doAs=bar&action=LIST&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 745865 INFO (qtp1912366258-17085) [n:127.0.0.1:44841_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={doAs=bar&action=LIST&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 745865 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[D0BA77A24A54D903]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1510342027763&s=nPUCQGroled/XN0VC0F89EAYVMM="; Path=/; Domain=127.0.0.1; Expires= , 10-11-2017 19:27:07 GMT; HttpOnly". Invalid 'expires' attribute: , 10-11-2017 19:27:07 GMT
[junit4] 2> 745865 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyValidateHost-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyValidateHost
[junit4] 2> 745878 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNoConfigGroups
[junit4] 2> 745879 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[D0BA77A24A54D903]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=noGroups&p=noGroups&t=test&e=1510342027777&s=VBGZh+L0mDFNoiHbFSNCNSj+hys="; Path=/; Domain=127.0.0.1; Expires= , 10-11-2017 19:27:07 GMT; HttpOnly". Invalid 'expires' attribute: , 10-11-2017 19:27:07 GMT
[junit4] 2> 745879 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNoConfigGroups-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNoConfigGroups
[junit4] 2> 745893 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyWrongHost
[junit4] 2> 746073 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[D0BA77A24A54D903]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=wrongHost&p=wrongHost&t=test&e=1510342027792&s=16RpqZ8hJA96SVr823hJm8qfu+Y="; Path=/; Domain=127.0.0.1; Expires= , 10-11-2017 19:27:07 GMT; HttpOnly". Invalid 'expires' attribute: , 10-11-2017 19:27:07 GMT
[junit4] 2> 746073 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyWrongHost-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyWrongHost
[junit4] 2> 746087 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNullProxyUser-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyNullProxyUser
[junit4] 2> 746087 INFO (qtp1912366258-17089) [n:127.0.0.1:44841_solr ] o.a.s.s.SolrDispatchFilter Error authenticating
[junit4] 2> java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamDelegationTokenAuthenticationHandler$HttpParamAuthenticationHandler.authenticate(HttpParamDelegationTokenPlugin.java:213)
[junit4] 2> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
[junit4] 2> at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
[junit4] 2> at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:134)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamToRequestFilter.doFilter(HttpParamDelegationTokenPlugin.java:245)
[junit4] 2> at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:243)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:451)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 746088 WARN (qtp1912366258-17089) [n:127.0.0.1:44841_solr ] o.e.j.s.ServletHandler /solr/admin/collections
[junit4] 2> org.apache.solr.common.SolrException: Error during request authentication,
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:457)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:347)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.lang.IllegalArgumentException: userName is NULL, empty or contains a '&'
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.checkForIllegalArgument(AuthToken.java:87)
[junit4] 2> at org.apache.hadoop.security.authentication.util.AuthToken.<init>(AuthToken.java:70)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationToken.<init>(AuthenticationToken.java:58)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamDelegationTokenAuthenticationHandler$HttpParamAuthenticationHandler.authenticate(HttpParamDelegationTokenPlugin.java:213)
[junit4] 2> at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.authenticate(DelegationTokenAuthenticationHandler.java:348)
[junit4] 2> at org.apache.solr.security.RequestContinuesRecorderAuthenticationHandler.authenticate(RequestContinuesRecorderAuthenticationHandler.java:69)
[junit4] 2> at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:518)
[junit4] 2> at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:134)
[junit4] 2> at org.apache.solr.security.HttpParamDelegationTokenPlugin$HttpParamToRequestFilter.doFilter(HttpParamDelegationTokenPlugin.java:245)
[junit4] 2> at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:243)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:451)
[junit4] 2> ... 26 more
[junit4] 2> 746089 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyNullProxyUser-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyNullProxyUser
[junit4] 2> 746100 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testProxyUnknownRemote
[junit4] 2> 757635 WARN (jetty-launcher-3221-thread-1-SendThread(127.0.0.1:36937)) [n:127.0.0.1:41609_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 14803ms for sessionid 0x15fa5410c100008
[junit4] 2> 757635 WARN (jetty-launcher-3221-thread-2-SendThread(127.0.0.1:36937)) [n:127.0.0.1:44841_solr ] o.a.z.ClientCnxn Client session timed out, have not heard from server in 14803ms for sessionid 0x15fa5410c100007
[junit4] 2> 757635 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15fa5410c100008, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 757635 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15fa5410c100007, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 757636 WARN (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[D0BA77A24A54D903]) [ ] o.a.h.c.p.ResponseProcessCookies Invalid cookie header: "Set-Cookie: hadoop.auth="u=localHostAnyGroup&p=localHostAnyGroup&t=test&e=1510342028000&s=RPQbuDAZus1VpAtTGrFOTxm7qgg="; Path=/; Domain=127.0.0.1; Expires= , 10-11-2017 19:27:08 GMT; HttpOnly". Invalid 'expires' attribute: , 10-11-2017 19:27:08 GMT
[junit4] 2> 757637 INFO (TEST-TestSolrCloudWithSecureImpersonation.testProxyUnknownRemote-seed#[D0BA77A24A54D903]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testProxyUnknownRemote
[junit4] 2> 757735 INFO (jetty-launcher-3221-thread-1-EventThread) [n:127.0.0.1:41609_solr ] o.a.c.f.s.ConnectionStateManager State change: SUSPENDED
[junit4] 2> 757737 INFO (jetty-closer-3222-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@128fe973{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 757737 INFO (jetty-closer-3222-thread-2) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@366d6954{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 757738 INFO (jetty-closer-3222-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=548615260
[junit4] 2> 757738 INFO (jetty-closer-3222-thread-2) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=536990190
[junit4] 2> 757738 INFO (jetty-closer-3222-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 757738 INFO (jetty-closer-3222-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 757738 INFO (jetty-closer-3222-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2db77a2c: rootName = solr_44841, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@63c1003c
[junit4] 2> 757739 INFO (jetty-closer-3222-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@3aeab20: rootName = solr_41609, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@32cad074
[junit4] 2> 757746 INFO (jetty-closer-3222-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 757746 INFO (jetty-closer-3222-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 757746 INFO (jetty-closer-3222-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@30515097: rootName = solr_41609, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@5a3f6f7e
[junit4] 2> 757746 INFO (jetty-closer-3222-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@44fb4564: rootName = solr_44841, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@5a3f6f7e
[junit4] 2> 757751 INFO (jetty-closer-3222-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 757751 INFO (jetty-closer-3222-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@19eacb98: rootName = solr_44841, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@139f43d1
[junit4] 2> 757752 INFO (jetty-closer-3222-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 757752 INFO (jetty-closer-3222-thread-2) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@69aef573: rootName = solr_41609, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@139f43d1
[junit4] 2> 757753 INFO (jetty-closer-3222-thread-1) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:44841_solr
[junit4] 2> 757753 INFO (jetty-closer-3222-thread-2) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:41609_solr
[junit4] 2> 757753 INFO (jetty-closer-3222-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 757754 INFO (jetty-closer-3222-thread-2) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 757754 INFO (zkCallback-3233-thread-2-processing-n:127.0.0.1:44841_solr) [n:127.0.0.1:44841_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (0)
[junit4] 2> 757754 INFO (zkCallback-3232-thread-1-processing-n:127.0.0.1:41609_solr) [n:127.0.0.1:41609_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (0)
[junit4] 2> 757754 INFO (coreCloseExecutor-3899-thread-1) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.c.SolrCore [forwardingCollection_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@2bbe89e1
[junit4] 2> 757754 INFO (jetty-closer-3222-thread-2) [ ] o.a.s.c.Overseer Overseer (id=98979415619272709-127.0.0.1:41609_solr-n_0000000000) closing
[junit4] 2> 757754 INFO (coreCloseExecutor-3899-thread-1) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.forwardingCollection.shard1.replica_n1, tag=733907425
[junit4] 2> 757754 INFO (OverseerStateUpdate-98979415619272709-127.0.0.1:41609_solr-n_0000000000) [n:127.0.0.1:41609_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:41609_solr
[junit4] 2> 757754 INFO (coreCloseExecutor-3899-thread-1) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@32bde395: rootName = solr_44841, domain = solr.core.forwardingCollection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.forwardingCollection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@4ae25b26
[junit4] 2> 757755 WARN (OverseerAutoScalingTriggerThread-98979415619272709-127.0.0.1:41609_solr-n_0000000000) [n:127.0.0.1:41609_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 757755 INFO (zkCallback-3233-thread-3-processing-n:127.0.0.1:44841_solr) [n:127.0.0.1:44841_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [0])
[junit4] 2> 757755 INFO (zkCallback-3233-thread-1-processing-n:127.0.0.1:44841_solr) [n:127.0.0.1:44841_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/forwardingCollection/state.json] for collection [forwardingCollection] has occurred - updating... (live nodes size: [0])
[junit4] 2> 757756 INFO (zkCallback-3233-thread-2-processing-n:127.0.0.1:44841_solr) [n:127.0.0.1:44841_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44841_solr
[junit4] 2> 757769 INFO (coreCloseExecutor-3899-thread-1) [n:127.0.0.1:44841_solr c:forwardingCollection s:shard1 r:core_node2 x:forwardingCollection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.forwardingCollection.shard1.leader, tag=733907425
[junit4] 2> 757771 ERROR (Thread[Thread-3591,5,TGRP-TestSolrCloudWithSecureImpersonation]) [n:127.0.0.1:44841_solr ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
[junit4] 2> 758629 INFO (jetty-closer-3222-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1a311aa9{/solr,null,UNAVAILABLE}
[junit4] 2> 759256 WARN (zkCallback-3232-thread-1-processing-n:127.0.0.1:41609_solr) [n:127.0.0.1:41609_solr ] o.a.s.c.ZkController Unable to register nodeLost path for 127.0.0.1:44841_solr
[junit4] 2> org.apache.zookeeper.KeeperException$SessionExpiredException: KeeperErrorCode = Session expired for /autoscaling/nodeLost/127.0.0.1:44841_solr
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:127)
[junit4] 2> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
[junit4] 2> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:783)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.lambda$create$7(SolrZkClient.java:357)
[junit4] 2> at org.apache.solr.common.cloud.ZkCmdExecutor.retryOperation(ZkCmdExecutor.java:60)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient.create(SolrZkClient.java:357)
[junit4] 2> at org.apache.solr.cloud.ZkController.lambda$registerLiveNodesListener$2(ZkController.java:879)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.lambda$refreshLiveNodes$1(ZkStateReader.java:749)
[junit4] 2> at java.util.concurrent.ConcurrentHashMap$KeySetView.forEach(ConcurrentHashMap.java:4649)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.refreshLiveNodes(ZkStateReader.java:748)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.access$1000(ZkStateReader.java:72)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.refreshAndWatch(ZkStateReader.java:1154)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader$LiveNodeWatcher.process(ZkStateReader.java:1149)
[junit4] 2> at org.apache.solr.common.cloud.SolrZkClient$1.lambda$process$0(SolrZkClient.java:262)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 759256 ERROR (Thread[Thread-3594,5,TGRP-TestSolrCloudWithSecureImpersonation]) [n:127.0.0.1:41609_solr ] o.a.h.s.t.d.AbstractDelegationTokenSecretManager ExpiredTokenRemover received java.lang.InterruptedException: sleep interrupted
[junit4] 2> 759257 INFO (jetty-closer-3222-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@36b3167e{/solr,null,UNAVAILABLE}
[junit4] 2> 759257 ERROR (SUITE-TestSolrCloudWithSecureImpersonation-seed#[D0BA77A24A54D903]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 759257 INFO (SUITE-TestSolrCloudWithSecureImpersonation-seed#[D0BA77A24A54D903]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36937 36937
[junit4] 2> 764432 INFO (Thread-3574) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:36937 36937
[junit4] 2> 764433 WARN (Thread-3574) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2> 2 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/collections/forwardingCollection/state.json
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMKeyIdRoot
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_1
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_3
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_2
[junit4] 2> 2 /solr/security/zkdtsm/ZKDTSMRoot/ZKDTSMMasterKeyRoot/DK_4
[junit4] 2> 2 /solr/security/zkdtsm/ZKD
[...truncated too long message...]
read-3) [n:127.0.0.1:41525_solr c:solrj_test_splitshard s:shard1_0 r:core_node7 x:solrj_test_splitshard_shard1_0_replica_n5] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.solrj_test_splitshard.shard1_0.leader, tag=1658230918
[junit4] 2> 1524923 INFO (coreCloseExecutor-7007-thread-4) [n:127.0.0.1:41525_solr c:solrj_test_splitshard s:shard1_1 r:core_node8 x:solrj_test_splitshard_shard1_1_replica_n6] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.solrj_test_splitshard.shard1_1.leader, tag=1268607619
[junit4] 2> 1524923 INFO (coreCloseExecutor-7007-thread-8) [n:127.0.0.1:41525_solr c:solrj_replicatests s:shard1 r:core_node3 x:solrj_replicatests_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.solrj_replicatests.shard1.leader, tag=730702315
[junit4] 2> 1524923 INFO (coreCloseExecutor-7007-thread-6) [n:127.0.0.1:41525_solr c:solrj_test_splitshard s:shard1_0_1 r:core_node13 x:solrj_test_splitshard_shard1_0_1_replica_n10] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.solrj_test_splitshard.shard1_0_1.leader, tag=2059668892
[junit4] 2> 1524923 INFO (coreCloseExecutor-7007-thread-10) [n:127.0.0.1:41525_solr c:solrj_test_core_props s:shard1 r:core_node2 x:solrj_test_core_props_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.solrj_test_core_props.shard1.leader, tag=1889769397
[junit4] 2> 1524923 INFO (coreCloseExecutor-7007-thread-14) [n:127.0.0.1:41525_solr c:replicaProperties s:shard1 r:core_node5 x:replicaProperties_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.replicaProperties.shard1.leader, tag=1515453589
[junit4] 2> 1526429 INFO (jetty-closer-2275-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2f031f82{/solr,null,UNAVAILABLE}
[junit4] 2> 1526429 ERROR (SUITE-CollectionsAPISolrJTest-seed#[D0BA77A24A54D903]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1526430 INFO (SUITE-CollectionsAPISolrJTest-seed#[D0BA77A24A54D903]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46047 46047
[junit4] 2> 1526576 INFO (Thread-3652) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46047 46047
[junit4] 2> 1526577 WARN (Thread-3652) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 9 /solr/collections/solrj_test/state.json
[junit4] 2> 8 /solr/configs/conf
[junit4] 2> 8 /solr/collections/corestatus_test/state.json
[junit4] 2> 5 /solr/aliases.json
[junit4] 2> 4 /solr/configs/solrj_default_configset
[junit4] 2> 4 /solr/configs/solrj_default_configset/managed-schema
[junit4] 2> 4 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 52 /solr/collections/solrj_implicit/state.json
[junit4] 2> 49 /solr/collections/solrj_test_splitshard/state.json
[junit4] 2> 33 /solr/collections/solrj_newstateformat/state.json
[junit4] 2> 29 /solr/collections/balancedProperties/state.json
[junit4] 2> 25 /solr/collections/replicaProperties/state.json
[junit4] 2> 24 /solr/collections/solrj_default_configset/state.json
[junit4] 2> 19 /solr/collections/solrj_replicatests/state.json
[junit4] 2> 5 /solr/collections/aliasedCollection/state.json
[junit4] 2> 5 /solr/clusterprops.json
[junit4] 2> 5 /solr/clusterstate.json
[junit4] 2> 5 /solr/collections/solrj_test_core_props/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 5 /solr/live_nodes
[junit4] 2> 5 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.cloud.CollectionsAPISolrJTest_D0BA77A24A54D903-001
[junit4] 2> Nov 10, 2017 9:40:08 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=true): {}, locale=hi, timezone=SystemV/AST4
[junit4] 2> NOTE: Linux 4.10.0-37-generic amd64/Oracle Corporation 1.8.0_144 (64-bit)/cpus=8,threads=1,free=176275016,total=518979584
[junit4] 2> NOTE: All tests run in this JVM: [DistribCursorPagingTest, V2StandaloneTest, CoreAdminRequestStatusTest, TestSolrConfigHandler, TestBackupRepositoryFactory, TestMacroExpander, TestReversedWildcardFilterFactory, TestLeaderElectionWithEmptyReplica, TestSchemaNameResource, TestBadConfig, TestLocalFSCloudBackupRestore, TestEmbeddedSolrServerSchemaAPI, TestConfigSetsAPI, TestComponentsName, TestPivotHelperCode, DateFieldTest, TestSolrFieldCacheBean, HdfsChaosMonkeySafeLeaderTest, TestRestManager, RAMDirectoryFactoryTest, DistributedQueryElevationComponentTest, TestSolrDeletionPolicy1, TestPHPSerializedResponseWriter, TestJoin, BasicZkTest, UpdateRequestProcessorFactoryTest, MultiTermTest, TestDFRSimilarityFactory, DistributedFacetPivotLargeTest, TestDistributedSearch, CdcrBidirectionalTest, TestStressCloudBlindAtomicUpdates, TestOrdValues, ClassificationUpdateProcessorTest, TestOnReconnectListenerSupport, PrimitiveFieldTypeTest, ChaosMonkeyNothingIsSafeWithPullReplicasTest, QueryParsingTest, SpellCheckCollatorTest, LeaderElectionIntegrationTest, TestCloudInspectUtil, HdfsUnloadDistributedZkTest, BigEndianAscendingWordSerializerTest, ResponseHeaderTest, TestHalfAndHalfDocValues, SpatialHeatmapFacetsTest, TestFoldingMultitermQuery, HdfsRecoverLeaseTest, ActionThrottleTest, TestPolicyCloud, InfixSuggestersTest, TermsComponentTest, MinimalSchemaTest, SimpleCollectionCreateDeleteTest, BasicAuthIntegrationTest, TestSolrIndexConfig, TestDownShardTolerantSearch, TestRequestStatusCollectionAPI, RemoteQueryErrorTest, TestSolrConfigHandlerConcurrent, ExternalFileFieldSortTest, LargeFieldTest, TestClusterStateMutator, TestFieldCacheVsDocValues, TestShortCircuitedRequests, TestRTimerTree, DocumentBuilderTest, TestFieldCacheReopen, TestRawTransformer, UpdateParamsTest, TestChildDocTransformer, TestCodecSupport, SaslZkACLProviderTest, DistributedQueueTest, SolrCoreTest, TestStressVersions, SortSpecParsingTest, DistributedDebugComponentTest, TestExportWriter, ConnectionManagerTest, TestInPlaceUpdatesDistrib, FileUtilsTest, TestHighlightDedupGrouping, SearchHandlerTest, TestSolrJ, HighlighterMaxOffsetTest, TestCollectionAPIs, BlobRepositoryCloudTest, FastVectorHighlighterTest, TestRebalanceLeaders, TestMergePolicyConfig, TestScoreJoinQPNoScore, SparseHLLTest, TestBinaryField, BadCopyFieldTest, LeaderInitiatedRecoveryOnShardRestartTest, TestSolrCloudWithHadoopAuthPlugin, DocValuesTest, TestReplicationHandler, TestAuthorizationFramework, TestSmileRequest, ConfigureRecoveryStrategyTest, TermVectorComponentTest, DateMathParserTest, SmileWriterTest, TestMaxScoreQueryParser, TestJavabinTupleStreamParser, SolrPluginUtilsTest, UninvertDocValuesMergePolicyTest, RestartWhileUpdatingTest, IndexSchemaRuntimeFieldTest, FullHLLTest, TestComplexPhraseLeadingWildcard, TestTrieFacet, TestSha256AuthenticationProvider, TestDynamicFieldCollectionResource, TestConfig, BlockJoinFacetDistribTest, DirectUpdateHandlerOptimizeTest, TestReloadAndDeleteDocs, TestAnalyzeInfixSuggestions, SyncSliceTest, ShardRoutingTest, RecoveryZkTest, ClusterStateUpdateTest, TestRandomFaceting, LeaderElectionTest, ZkCLITest, ShardRoutingCustomTest, TestFaceting, TestRecovery, TestRangeQuery, QueryElevationComponentTest, PeerSyncTest, SolrIndexSplitterTest, SimplePostToolTest, AnalysisAfterCoreReloadTest, SignatureUpdateProcessorFactoryTest, SuggesterFSTTest, SolrRequestParserTest, SuggesterTSTTest, SuggesterTest, TestStressLucene, SpatialFilterTest, TestCSVLoader, NoCacheHeaderTest, TestPseudoReturnFields, TestUpdate, TestWordDelimiterFilterFactory, QueryEqualityTest, StatelessScriptUpdateProcessorFactoryTest, TestRemoteStreaming, DocValuesMultiTest, SolrInfoBeanTest, XsltUpdateRequestHandlerTest, CacheHeaderTest, TestSurroundQueryParser, LukeRequestHandlerTest, DisMaxRequestHandlerTest, TestQueryUtils, TestWriterPerf, TestOmitPositions, FileBasedSpellCheckerTest, XmlUpdateRequestHandlerTest, TestIndexingPerformance, FieldAnalysisRequestHandlerTest, RegexBoostProcessorTest, TestAnalyzedSuggestions, JsonLoaderTest, CSVRequestHandlerTest, HighlighterConfigTest, SOLR749Test, AlternateDirectoryTest, CopyFieldTest, ResponseLogComponentTest, SolrIndexConfigTest, TestSolrDeletionPolicy2, TestDocSet, TestSearchPerf, NumericFieldsTest, TestElisionMultitermQuery, OutputWriterTest, TestFuzzyAnalyzedSuggestions, TestPostingsSolrHighlighter, TestPhraseSuggestions, SpellPossibilityIteratorTest, TestXIncludeConfig, TestSweetSpotSimilarityFactory, TestIBSimilarityFactory, TestBM25SimilarityFactory, TestFastWriter, TimeZoneUtilsTest, ResourceLoaderTest, TestFastOutputStream, ScriptEngineTest, URLClassifyProcessorTest, PluginInfoTest, ChaosMonkeyNothingIsSafeTest, TestSuggestSpellingConverter, SpellingQueryConverterTest, DOMUtilTest, TestLRUCache, TestUtils, TestDocumentBuilder, ZkNodePropsTest, SliceStateTest, SystemInfoHandlerTest, UUIDFieldTest, DistributedMLTComponentTest, TestRTGBase, CursorPagingTest, DistributedIntervalFacetingTest, TestEmbeddedSolrServerAdminHandler, TestEmbeddedSolrServerConstructors, TestJettySolrRunner, ConnectionReuseTest, AssignBackwardCompatibilityTest, CollectionStateFormat2Test, CollectionsAPISolrJTest]
[junit4] Completed [453/748 (3!)] on J0 in 145.00s, 13 tests, 1 failure <<< FAILURES!
[...truncated 48285 lines...]
[JENKINS] Lucene-Solr-7.x-Linux (32bit/jdk1.8.0_144) - Build # 785
- Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/785/
Java: 32bit/jdk1.8.0_144 -server -XX:+UseParallelGC
2 tests failed.
FAILED: org.apache.solr.security.TestAuthorizationFramework.authorizationFrameworkTest
Error Message:
Timeout occured while waiting response from server at: http://127.0.0.1:37771/c
Stack Trace:
org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:37771/c
at __randomizedtesting.SeedInfo.seed([C7920C71E5634973:893179A2F4B85863]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:883)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:314)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
... 43 more
FAILED: org.apache.solr.client.solrj.impl.CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale
Error Message:
Error from server at https://127.0.0.1:44181/solr/stale_state_test_col_shard1_replica_n1: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 404 </title> </head> <body> <h2>HTTP ERROR: 404</h2> <p>Problem accessing /solr/stale_state_test_col_shard1_replica_n1/update. Reason: <pre> Can not find: /solr/stale_state_test_col_shard1_replica_n1/update</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.20.v20170531</a><hr/> </body> </html>
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:44181/solr/stale_state_test_col_shard1_replica_n1: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 </title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /solr/stale_state_test_col_shard1_replica_n1/update. Reason:
<pre> Can not find: /solr/stale_state_test_col_shard1_replica_n1/update</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.20.v20170531</a><hr/>
</body>
</html>
at __randomizedtesting.SeedInfo.seed([4470578693723F30:F041CF6E709B491C]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:559)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1016)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:883)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
at org.apache.solr.client.solrj.impl.CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale(CloudSolrClientTest.java:844)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12408 lines...]
[junit4] Suite: org.apache.solr.security.TestAuthorizationFramework
[junit4] 2> 880672 INFO (SUITE-TestAuthorizationFramework-seed#[C7920C71E5634973]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.security.TestAuthorizationFramework_C7920C71E5634973-001/init-core-data-001
[junit4] 2> 880672 WARN (SUITE-TestAuthorizationFramework-seed#[C7920C71E5634973]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=4 numCloses=4
[junit4] 2> 880672 INFO (SUITE-TestAuthorizationFramework-seed#[C7920C71E5634973]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 880673 INFO (SUITE-TestAuthorizationFramework-seed#[C7920C71E5634973]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 880673 INFO (SUITE-TestAuthorizationFramework-seed#[C7920C71E5634973]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /c/
[junit4] 2> 880674 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 880674 INFO (Thread-1056) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 880675 INFO (Thread-1056) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 880675 ERROR (Thread-1056) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 880775 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.ZkTestServer start zk server on port:34327
[junit4] 2> 880789 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 880790 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 880790 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 880791 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 880791 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 880792 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 880792 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 880792 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 880793 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 880793 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 880794 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 880794 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 880890 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 880893 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1a7d3c7{/c,null,AVAILABLE}
[junit4] 2> 880894 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2d530d{HTTP/1.1,[http/1.1]}{127.0.0.1:37771}
[junit4] 2> 880894 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.e.j.s.Server Started @882471ms
[junit4] 2> 880894 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.security.TestAuthorizationFramework_C7920C71E5634973-001/tempDir-001/control/data, hostContext=/c, hostPort=37771, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.security.TestAuthorizationFramework_C7920C71E5634973-001/control-001/cores}
[junit4] 2> 880895 ERROR (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 880895 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.2.0
[junit4] 2> 880895 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 880895 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 880895 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-10T06:36:34.991Z
[junit4] 2> 880899 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 880899 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.security.TestAuthorizationFramework_C7920C71E5634973-001/control-001/solr.xml
[junit4] 2> 880903 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 880903 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 880903 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@18c9915, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 880907 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34327/solr
[junit4] 2> 880951 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [n:127.0.0.1:37771_c ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 880951 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [n:127.0.0.1:37771_c ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:37771_c
[junit4] 2> 880952 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [n:127.0.0.1:37771_c ] o.a.s.c.Overseer Overseer (id=98978745206112261-127.0.0.1:37771_c-n_0000000000) starting
[junit4] 2> 880955 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [n:127.0.0.1:37771_c ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:37771_c
[junit4] 2> 880969 INFO (OverseerStateUpdate-98978745206112261-127.0.0.1:37771_c-n_0000000000) [n:127.0.0.1:37771_c ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 881052 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [n:127.0.0.1:37771_c ] o.a.s.c.CoreContainer Initializing authorization plugin: org.apache.solr.security.MockAuthorizationPlugin
[junit4] 2> 881076 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [n:127.0.0.1:37771_c ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@18c9915
[junit4] 2> 881082 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [n:127.0.0.1:37771_c ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@18c9915
[junit4] 2> 881082 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [n:127.0.0.1:37771_c ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@18c9915
[junit4] 2> 881083 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [n:127.0.0.1:37771_c ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.security.TestAuthorizationFramework_C7920C71E5634973-001/control-001/cores
[junit4] 2> 881096 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 881097 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34327/solr ready
[junit4] 2> 881098 INFO (qtp2406209-6049) [n:127.0.0.1:37771_c ] o.a.s.s.MockAuthorizationPlugin User request: null
[junit4] 2> 881098 INFO (qtp2406209-6049) [n:127.0.0.1:37771_c ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:37771_c&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 881099 INFO (OverseerThreadFactory-2168-thread-1-processing-n:127.0.0.1:37771_c) [n:127.0.0.1:37771_c ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 881202 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c ] o.a.s.s.MockAuthorizationPlugin User request: null
[junit4] 2> 881203 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 881203 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 881305 INFO (zkCallback-769-thread-1-processing-n:127.0.0.1:37771_c) [n:127.0.0.1:37771_c ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 882228 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.2.0
[junit4] 2> 882239 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 882294 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 882302 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 882302 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@18c9915
[junit4] 2> 882302 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 882302 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.security.TestAuthorizationFramework_C7920C71E5634973-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.security.TestAuthorizationFramework_C7920C71E5634973-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 882304 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=44, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.7809602351686146]
[junit4] 2> 882306 WARN (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 882332 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 882332 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 882333 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 882333 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 882334 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=37, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.15168137046855634]
[junit4] 2> 882334 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@c859c4[control_collection_shard1_replica_n1] main]
[junit4] 2> 882335 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 882335 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 882336 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 882337 INFO (searcherExecutor-2171-thread-1-processing-n:127.0.0.1:37771_c x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@c859c4[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 882337 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1583659925040529408
[junit4] 2> 882339 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 882339 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 882340 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:37771/c/control_collection_shard1_replica_n1/
[junit4] 2> 882340 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 882340 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:37771/c/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 882340 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 882341 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:37771/c/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 941154 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.ChaosMonkey monkey: stop jetty! 37771
[junit4] 2> 941154 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=11649854
[junit4] 2> 941154 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 941154 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1ecb1b1: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@17fe73e
[junit4] 2> 941158 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 941158 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@bdbfd4: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@dd1511
[junit4] 2> 941161 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 941161 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@be045d: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@2280aa
[junit4] 2> 941162 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:37771_c
[junit4] 2> 941162 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 941162 INFO (zkCallback-769-thread-2-processing-n:127.0.0.1:37771_c) [n:127.0.0.1:37771_c ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 941163 INFO (coreCloseExecutor-2176-thread-1) [n:127.0.0.1:37771_c c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@1513a18
[junit4] 2> 941163 INFO (coreCloseExecutor-2176-thread-1) [n:127.0.0.1:37771_c c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.control_collection.shard1.replica_n1, tag=22100504
[junit4] 2> 941163 INFO (coreCloseExecutor-2176-thread-1) [n:127.0.0.1:37771_c c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@481a78: rootName = null, domain = solr.core.control_collection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.control_collection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@aa06f
[junit4] 2> 941168 INFO (coreCloseExecutor-2176-thread-1) [n:127.0.0.1:37771_c c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.control_collection.shard1.leader, tag=22100504
[junit4] 2> 951971 INFO (ScheduledTrigger-2166-thread-1) [n:127.0.0.1:37771_c ] o.a.s.c.a.SystemLogListener Collection .system does not exist, disabling logging.
[junit4] 2> 951971 INFO (AutoscalingActionExecutor-2167-thread-1-processing-n:127.0.0.1:37771_c) [n:127.0.0.1:37771_c ] o.a.s.c.a.ExecutePlanAction No operations to execute for event: {
[junit4] 2> "id":"14f5a6144651eb80Tbvmdjjsxgca2ctzfqtv9mqrbe",
[junit4] 2> "source":".auto_add_replicas",
[junit4] 2> "eventTime":1510295856062000000,
[junit4] 2> "eventType":"NODELOST",
[junit4] 2> "properties":{
[junit4] 2> "eventTimes":[1510295856062000000],
[junit4] 2> "_enqueue_time_":1510295866066000000,
[junit4] 2> "nodeNames":["127.0.0.1:37771_c"]}}
[junit4] 2> 1061100 ERROR (qtp2406209-6049) [n:127.0.0.1:37771_c ] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: create the collection time out:180s
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:312)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:244)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:222)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:737)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:718)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:499)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 1061101 INFO (qtp2406209-6049) [n:127.0.0.1:37771_c ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:37771_c&wt=javabin&version=2} status=500 QTime=180003
[junit4] 2> 1061101 ERROR (qtp2406209-6049) [n:127.0.0.1:37771_c ] o.a.s.s.HttpSolrCall null:org.apache.solr.common.SolrException: create the collection time out:180s
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:312)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:244)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:222)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:737)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:718)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:499)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 1481298 ERROR (OverseerThreadFactory-2168-thread-1-processing-n:127.0.0.1:37771_c) [n:127.0.0.1:37771_c ] o.a.s.c.OverseerCollectionMessageHandler Error from shard: http://127.0.0.1:37771/c
[junit4] 2> org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:37771/c
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] 2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
[junit4] 2> at org.apache.solr.handler.component.HttpShardHandler.lambda$submit$0(HttpShardHandler.java:172)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
[junit4] 2> at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: java.net.SocketTimeoutException: Read timed out
[junit4] 2> at java.net.SocketInputStream.socketRead0(Native Method)
[junit4] 2> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
[junit4] 2> at java.net.SocketInputStream.read(SocketInputStream.java:171)
[junit4] 2> at java.net.SocketInputStream.read(SocketInputStream.java:141)
[junit4] 2> at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
[junit4] 2> at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
[junit4] 2> at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
[junit4] 2> at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
[junit4] 2> at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
[junit4] 2> at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
[junit4] 2> at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
[junit4] 2> at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
[junit4] 2> at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
[junit4] 2> at org.apache.solr.util.stats.InstrumentedHttpRequestExecutor.execute(InstrumentedHttpRequestExecutor.java:118)
[junit4] 2> at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
[junit4] 2> at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
[junit4] 2> at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
[junit4] 2> at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
[junit4] 2> at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
[junit4] 2> at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
[junit4] 2> ... 12 more
[junit4] 2> 1481298 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.Overseer Overseer (id=98978745206112261-127.0.0.1:37771_c-n_0000000000) closing
[junit4] 2> 1481300 ERROR (OverseerThreadFactory-2168-thread-1-processing-n:127.0.0.1:37771_c) [n:127.0.0.1:37771_c ] o.a.s.c.OverseerCollectionMessageHandler Cleaning up collection [control_collection].
[junit4] 2> 1481300 INFO (OverseerStateUpdate-98978745206112261-127.0.0.1:37771_c-n_0000000000) [n:127.0.0.1:37771_c ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:37771_c
[junit4] 2> 1481301 INFO (OverseerThreadFactory-2168-thread-1-processing-n:127.0.0.1:37771_c) [n:127.0.0.1:37771_c ] o.a.s.c.OverseerCollectionMessageHandler Executing Collection Cmd : action=UNLOAD&deleteInstanceDir=true&deleteDataDir=true
[junit4] 2> 1511334 INFO (zkCallback-769-thread-5-processing-n:127.0.0.1:37771_c) [n:127.0.0.1:37771_c ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDeleted path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [0])
[junit4] 2> 1511335 ERROR (OverseerThreadFactory-2168-thread-1-processing-n:127.0.0.1:37771_c) [n:127.0.0.1:37771_c ] o.a.s.c.OverseerCollectionMessageHandler Collection: control_collection operation: create failed:org.apache.solr.common.SolrException: Could not fully remove collection: control_collection
[junit4] 2> at org.apache.solr.cloud.DeleteCollectionCmd.call(DeleteCollectionCmd.java:109)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.cleanupCollection(OverseerCollectionMessageHandler.java:678)
[junit4] 2> at org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:302)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:232)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:464)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 1511341 INFO (OverseerThreadFactory-2168-thread-1-processing-n:127.0.0.1:37771_c) [n:127.0.0.1:37771_c ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1511341 WARN (OverseerAutoScalingTriggerThread-98978745206112261-127.0.0.1:37771_c-n_0000000000) [n:127.0.0.1:37771_c ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 1511342 ERROR (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController Error getting leader from zk
[junit4] 2> org.apache.solr.common.SolrException: No registered leader was found after waiting for 1220000ms , collection: control_collection slice: shard1 saw state=null with live_nodes=[]
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:816)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:781)
[junit4] 2> at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1178)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1079)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1014)
[junit4] 2> at org.apache.solr.core.ZkContainer.lambda$registerInZk$0(ZkContainer.java:181)
[junit4] 2> at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:208)
[junit4] 2> at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:886)
[junit4] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1047)
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:948)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:737)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:718)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:499)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1511343 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@2d530d{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 1511343 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1a7d3c7{/c,null,UNAVAILABLE}
[junit4] 2> 1511343 ERROR (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkContainer :org.apache.solr.common.SolrException: Error getting leader from zk for shard shard1
[junit4] 2> at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1207)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1079)
[junit4] 2> at org.apache.solr.cloud.ZkController.register(ZkController.java:1014)
[junit4] 2> at org.apache.solr.core.ZkContainer.lambda$registerInZk$0(ZkContainer.java:181)
[junit4] 2> at org.apache.solr.core.ZkContainer.registerInZk(ZkContainer.java:208)
[junit4] 2> at org.apache.solr.core.CoreContainer.registerCore(CoreContainer.java:886)
[junit4] 2> at org.apache.solr.core.CoreContainer.createFromDescriptor(CoreContainer.java:1047)
[junit4] 2> at org.apache.solr.core.CoreContainer.create(CoreContainer.java:948)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.lambda$static$0(CoreAdminOperation.java:91)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminOperation.execute(CoreAdminOperation.java:384)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:389)
[junit4] 2> at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:174)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:737)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:718)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:499)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:426)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> Caused by: org.apache.solr.common.SolrException: No registered leader was found after waiting for 1220000ms , collection: control_collection slice: shard1 saw state=null with live_nodes=[]
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderRetry(ZkStateReader.java:816)
[junit4] 2> at org.apache.solr.common.cloud.ZkStateReader.getLeaderUrl(ZkStateReader.java:781)
[junit4] 2> at org.apache.solr.cloud.ZkController.getLeader(ZkController.java:1178)
[junit4] 2> ... 41 more
[junit4] 2>
[junit4] 2> 1511344 INFO (qtp2406209-6051) [n:127.0.0.1:37771_c c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=630141
[junit4] 2> 1511345 ERROR (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1511347 INFO (TEST-TestAuthorizationFramework.authorizationFrameworkTest-seed#[C7920C71E5634973]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34327 34327
[junit4] 2> 1514530 INFO (Thread-1056) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34327 34327
[junit4] 2> 1514531 WARN (Thread-1056) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/clusterstate.json
[junit4] 2> 2 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/live_nodes
[junit4] 2> 2 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestAuthorizationFramework -Dtests.method=authorizationFrameworkTest -Dtests.seed=C7920C71E5634973 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=ar-YE -Dtests.timezone=Europe/Rome -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] ERROR 634s J0 | TestAuthorizationFramework.authorizationFrameworkTest <<<
[junit4] > Throwable #1: org.apache.solr.client.solrj.SolrServerException: Timeout occured while waiting response from server at: http://127.0.0.1:37771/c
[junit4] > at __randomizedtesting.SeedInfo.seed([C7920C71E5634973:893179A2F4B85863]:0)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:654)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:883)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:314)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] > Caused by: java.net.SocketTimeoutException: Read timed out
[junit4] > at java.net.SocketInputStream.socketRead0(Native Method)
[junit4] > at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
[junit4] > at java.net.SocketInputStream.read(SocketInputStream.java:171)
[junit4] > at java.net.SocketInputStream.read(SocketInputStream.java:141)
[junit4] > at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137)
[junit4] > at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153)
[junit4] > at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:282)
[junit4] > at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138)
[junit4] > at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
[junit4] > at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
[junit4] > at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
[junit4] > at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:165)
[junit4] > at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
[junit4] > at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
[junit4] > at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
[junit4] > at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:185)
[junit4] > at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
[junit4] > at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:111)
[junit4] > at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
[junit4] > at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
[junit4] > at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:56)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:542)
[junit4] > ... 43 more
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.security.TestAuthorizationFramework_C7920C71E5634973-001
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=true): {}, locale=ar-YE, timezone=Europe/Rome
[junit4] 2> NOTE: Linux 4.10.0-37-generic i386/Oracle Corporation 1.8.0_144 (32-bit)/cpus=8,threads=1,free=55431616,total=345243648
[junit4] 2> NOTE: All tests run in this JVM: [TestSha256AuthenticationProvider, MetricsHandlerTest, OutOfBoxZkACLAndCredentialsProvidersTest, UpdateRequestProcessorFactoryTest, CloudMLTQParserTest, DeleteReplicaTest, MinimalSchemaTest, ActionThrottleTest, TestTlogReplica, TestChildDocTransformer, HdfsRecoverLeaseTest, CoreAdminRequestStatusTest, TestHighlightDedupGrouping, TestCryptoKeys, TestRawTransformer, SortSpecParsingTest, TestLocalFSCloudBackupRestore, TestInPlaceUpdatesDistrib, TestBlobHandler, NumberUtilsTest, TestExportWriter, DistribCursorPagingTest, BasicFunctionalityTest, ExitableDirectoryReaderTest, SolrPluginUtilsTest, AtomicUpdatesTest, DistributedQueryComponentOptimizationTest, IndexSchemaRuntimeFieldTest, TestMacroExpander, SpellCheckComponentTest, TestBinaryField, DateMathParserTest, TestSolrDeletionPolicy1, TermsComponentTest, BasicAuthIntegrationTest, LeaderFailoverAfterPartitionTest, TestPayloadScoreQParserPlugin, TestFieldResource, CurrencyFieldTypeTest, TestSuggestSpellingConverter, TestReversedWildcardFilterFactory, StatelessScriptUpdateProcessorFactoryTest, TestBadConfig, TestSort, ForceLeaderTest, TestSolrCloudWithDelegationTokens, TestFieldSortValues, TestFieldCacheVsDocValues, TolerantUpdateProcessorTest, TestOrdValues, ResponseHeaderTest, TestCSVLoader, BasicZkTest, ExternalFileFieldSortTest, TestScoreJoinQPNoScore, ZkFailoverTest, TestAuthorizationFramework]
[junit4] Completed [245/748 (1!)] on J0 in 633.88s, 1 test, 1 error <<< FAILURES!
[...truncated 2363 lines...]
[junit4] Suite: org.apache.solr.client.solrj.impl.CloudSolrClientTest
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.impl.CloudSolrClientTest_4470578693723F30-001/init-core-data-001
[junit4] 2> 38963 INFO (SUITE-CloudSolrClientTest-seed#[4470578693723F30]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 38963 INFO (SUITE-CloudSolrClientTest-seed#[4470578693723F30]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 38964 INFO (SUITE-CloudSolrClientTest-seed#[4470578693723F30]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 38965 INFO (SUITE-CloudSolrClientTest-seed#[4470578693723F30]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 3 servers in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.impl.CloudSolrClientTest_4470578693723F30-001/tempDir-001
[junit4] 2> 38966 INFO (SUITE-CloudSolrClientTest-seed#[4470578693723F30]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 38968 INFO (Thread-72) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 38968 INFO (Thread-72) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 38980 ERROR (Thread-72) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 39068 INFO (SUITE-CloudSolrClientTest-seed#[4470578693723F30]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:46709
[junit4] 2> 39074 INFO (jetty-launcher-69-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 39074 INFO (jetty-launcher-69-thread-2) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 39082 INFO (jetty-launcher-69-thread-3) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 39082 INFO (jetty-launcher-69-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4a945e{/solr,null,AVAILABLE}
[junit4] 2> 39083 INFO (jetty-launcher-69-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@11b27d8{/solr,null,AVAILABLE}
[junit4] 2> 39085 INFO (jetty-launcher-69-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1d89b7b{SSL,[ssl, http/1.1]}{127.0.0.1:43939}
[junit4] 2> 39085 INFO (jetty-launcher-69-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@8940dd{SSL,[ssl, http/1.1]}{127.0.0.1:43351}
[junit4] 2> 39085 INFO (jetty-launcher-69-thread-3) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@20306e{/solr,null,AVAILABLE}
[junit4] 2> 39085 INFO (jetty-launcher-69-thread-1) [ ] o.e.j.s.Server Started @41405ms
[junit4] 2> 39085 INFO (jetty-launcher-69-thread-2) [ ] o.e.j.s.Server Started @41405ms
[junit4] 2> 39085 INFO (jetty-launcher-69-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43351}
[junit4] 2> 39085 INFO (jetty-launcher-69-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43939}
[junit4] 2> 39085 ERROR (jetty-launcher-69-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 39085 ERROR (jetty-launcher-69-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 39085 INFO (jetty-launcher-69-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.2.0
[junit4] 2> 39085 INFO (jetty-launcher-69-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.2.0
[junit4] 2> 39085 INFO (jetty-launcher-69-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 39085 INFO (jetty-launcher-69-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 39085 INFO (jetty-launcher-69-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 39085 INFO (jetty-launcher-69-thread-3) [ ] o.e.j.s.AbstractConnector Started ServerConnector@11b0ceb{SSL,[ssl, http/1.1]}{127.0.0.1:44181}
[junit4] 2> 39085 INFO (jetty-launcher-69-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-10T07:20:03.622Z
[junit4] 2> 39085 INFO (jetty-launcher-69-thread-3) [ ] o.e.j.s.Server Started @41406ms
[junit4] 2> 39085 INFO (jetty-launcher-69-thread-3) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=44181}
[junit4] 2> 39085 ERROR (jetty-launcher-69-thread-3) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 39085 INFO (jetty-launcher-69-thread-3) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.2.0
[junit4] 2> 39085 INFO (jetty-launcher-69-thread-3) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 39085 INFO (jetty-launcher-69-thread-3) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 39085 INFO (jetty-launcher-69-thread-3) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-10T07:20:03.622Z
[junit4] 2> 39085 INFO (jetty-launcher-69-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 39086 INFO (jetty-launcher-69-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-10T07:20:03.623Z
[junit4] 2> 39097 INFO (jetty-launcher-69-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 39099 INFO (jetty-launcher-69-thread-3) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 39103 INFO (jetty-launcher-69-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 39121 INFO (jetty-launcher-69-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46709/solr
[junit4] 2> 39151 INFO (jetty-launcher-69-thread-3) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46709/solr
[junit4] 2> 39152 INFO (jetty-launcher-69-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:46709/solr
[junit4] 2> 39215 WARN (jetty-launcher-69-thread-3) [n:127.0.0.1:44181_solr ] o.a.s.c.ZkController The _default configset could not be uploaded. Please provide 'solr.default.confdir' parameter that points to a configset intended to be the default. Current 'solr.default.confdir' value: null
[junit4] 2> 39215 WARN (jetty-launcher-69-thread-2) [n:127.0.0.1:43939_solr ] o.a.s.c.ZkController The _default configset could not be uploaded. Please provide 'solr.default.confdir' parameter that points to a configset intended to be the default. Current 'solr.default.confdir' value: null
[junit4] 2> 39219 WARN (jetty-launcher-69-thread-1) [n:127.0.0.1:43351_solr ] o.a.s.c.ZkController The _default configset could not be uploaded. Please provide 'solr.default.confdir' parameter that points to a configset intended to be the default. Current 'solr.default.confdir' value: null
[junit4] 2> 39220 INFO (jetty-launcher-69-thread-3) [n:127.0.0.1:44181_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 39220 INFO (jetty-launcher-69-thread-2) [n:127.0.0.1:43939_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 39223 INFO (jetty-launcher-69-thread-3) [n:127.0.0.1:44181_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:44181_solr
[junit4] 2> 39224 INFO (jetty-launcher-69-thread-3) [n:127.0.0.1:44181_solr ] o.a.s.c.Overseer Overseer (id=98978916172562440-127.0.0.1:44181_solr-n_0000000000) starting
[junit4] 2> 39224 INFO (jetty-launcher-69-thread-2) [n:127.0.0.1:43939_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43939_solr
[junit4] 2> 39229 INFO (zkCallback-85-thread-1-processing-n:127.0.0.1:44181_solr) [n:127.0.0.1:44181_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 39229 INFO (zkCallback-84-thread-1-processing-n:127.0.0.1:43351_solr) [n:127.0.0.1:43351_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 39235 INFO (zkCallback-86-thread-1-processing-n:127.0.0.1:43939_solr) [n:127.0.0.1:43939_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 39235 INFO (jetty-launcher-69-thread-1) [n:127.0.0.1:43351_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 39237 INFO (jetty-launcher-69-thread-1) [n:127.0.0.1:43351_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43351_solr
[junit4] 2> 39239 INFO (zkCallback-86-thread-1-processing-n:127.0.0.1:43939_solr) [n:127.0.0.1:43939_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 39243 INFO (zkCallback-84-thread-1-processing-n:127.0.0.1:43351_solr) [n:127.0.0.1:43351_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 39244 INFO (zkCallback-85-thread-1-processing-n:127.0.0.1:44181_solr) [n:127.0.0.1:44181_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 39253 INFO (jetty-launcher-69-thread-3) [n:127.0.0.1:44181_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44181_solr
[junit4] 2> 39254 INFO (zkCallback-84-thread-1-processing-n:127.0.0.1:43351_solr) [n:127.0.0.1:43351_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 39254 INFO (zkCallback-85-thread-1-processing-n:127.0.0.1:44181_solr) [n:127.0.0.1:44181_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 39254 INFO (zkCallback-86-thread-1-processing-n:127.0.0.1:43939_solr) [n:127.0.0.1:43939_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (3)
[junit4] 2> 39338 INFO (jetty-launcher-69-thread-1) [n:127.0.0.1:43351_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43351.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@e9b5b8
[junit4] 2> 39345 INFO (jetty-launcher-69-thread-1) [n:127.0.0.1:43351_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43351.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@e9b5b8
[junit4] 2> 39345 INFO (jetty-launcher-69-thread-1) [n:127.0.0.1:43351_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43351.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@e9b5b8
[junit4] 2> 39346 INFO (jetty-launcher-69-thread-1) [n:127.0.0.1:43351_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.impl.CloudSolrClientTest_4470578693723F30-001/tempDir-001/node1/.
[junit4] 2> 39388 INFO (jetty-launcher-69-thread-2) [n:127.0.0.1:43939_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43939.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@e9b5b8
[junit4] 2> 39395 INFO (jetty-launcher-69-thread-2) [n:127.0.0.1:43939_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43939.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@e9b5b8
[junit4] 2> 39395 INFO (jetty-launcher-69-thread-2) [n:127.0.0.1:43939_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43939.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@e9b5b8
[junit4] 2> 39397 INFO (jetty-launcher-69-thread-2) [n:127.0.0.1:43939_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.impl.CloudSolrClientTest_4470578693723F30-001/tempDir-001/node2/.
[junit4] 2> 39407 INFO (jetty-launcher-69-thread-3) [n:127.0.0.1:44181_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44181.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@e9b5b8
[junit4] 2> 39414 INFO (jetty-launcher-69-thread-3) [n:127.0.0.1:44181_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44181.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@e9b5b8
[junit4] 2> 39414 INFO (jetty-launcher-69-thread-3) [n:127.0.0.1:44181_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_44181.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@e9b5b8
[junit4] 2> 39416 INFO (jetty-launcher-69-thread-3) [n:127.0.0.1:44181_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.impl.CloudSolrClientTest_4470578693723F30-001/tempDir-001/node3/.
[junit4] 2> 39502 INFO (SUITE-CloudSolrClientTest-seed#[4470578693723F30]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (3)
[junit4] 2> 39502 INFO (SUITE-CloudSolrClientTest-seed#[4470578693723F30]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:46709/solr ready
[junit4] 2> 39586 INFO (qtp16972393-409) [n:127.0.0.1:43939_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 39590 INFO (OverseerThreadFactory-155-thread-1-processing-n:127.0.0.1:44181_solr) [n:127.0.0.1:44181_solr ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 39695 INFO (OverseerStateUpdate-98978916172562440-127.0.0.1:44181_solr-n_0000000000) [n:127.0.0.1:44181_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:43351/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 39697 INFO (OverseerStateUpdate-98978916172562440-127.0.0.1:44181_solr-n_0000000000) [n:127.0.0.1:44181_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"collection1_shard2_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:43939/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 39929 INFO (qtp16972393-407) [n:127.0.0.1:43939_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=collection1_shard2_replica_n2&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 39929 INFO (qtp16972393-407) [n:127.0.0.1:43939_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 39973 INFO (qtp28508859-400) [n:127.0.0.1:43351_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=collection1_shard1_replica_n1&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 39974 INFO (qtp28508859-400) [n:127.0.0.1:43351_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 40076 INFO (zkCallback-84-thread-1-processing-n:127.0.0.1:43351_solr) [n:127.0.0.1:43351_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [Watch
[...truncated too long message...]
der A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/foo/state.json] for collection [foo] has occurred - updating... (live nodes size: [0])
[junit4] 2> 73290 INFO (zkCallback-85-thread-3-processing-n:127.0.0.1:44181_solr) [n:127.0.0.1:44181_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/foo/state.json] for collection [foo] has occurred - updating... (live nodes size: [0])
[junit4] 2> 73289 INFO (zkCallback-84-thread-1-processing-n:127.0.0.1:43351_solr) [n:127.0.0.1:43351_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/foo/state.json] for collection [foo] has occurred - updating... (live nodes size: [0])
[junit4] 2> 73293 INFO (zkCallback-86-thread-1-processing-n:127.0.0.1:43939_solr) [n:127.0.0.1:43939_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/multicollection1/state.json] for collection [multicollection1] has occurred - updating... (live nodes size: [0])
[junit4] 2> 73294 INFO (zkCallback-86-thread-3-processing-n:127.0.0.1:43939_solr) [n:127.0.0.1:43939_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/multicollection1/state.json] for collection [multicollection1] has occurred - updating... (live nodes size: [0])
[junit4] 2> 73294 INFO (zkCallback-84-thread-2-processing-n:127.0.0.1:43351_solr) [n:127.0.0.1:43351_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/multicollection1/state.json] for collection [multicollection1] has occurred - updating... (live nodes size: [0])
[junit4] 2> 73294 INFO (zkCallback-84-thread-1-processing-n:127.0.0.1:43351_solr) [n:127.0.0.1:43351_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/multicollection1/state.json] for collection [multicollection1] has occurred - updating... (live nodes size: [0])
[junit4] 2> 73294 INFO (zkCallback-85-thread-3-processing-n:127.0.0.1:44181_solr) [n:127.0.0.1:44181_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/multicollection2/state.json] for collection [multicollection2] has occurred - updating... (live nodes size: [0])
[junit4] 2> 73294 INFO (zkCallback-86-thread-3-processing-n:127.0.0.1:43939_solr) [n:127.0.0.1:43939_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/multicollection2/state.json] for collection [multicollection2] has occurred - updating... (live nodes size: [0])
[junit4] 2> 73294 INFO (zkCallback-85-thread-1-processing-n:127.0.0.1:44181_solr) [n:127.0.0.1:44181_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/multicollection2/state.json] for collection [multicollection2] has occurred - updating... (live nodes size: [0])
[junit4] 2> 73294 INFO (zkCallback-86-thread-1-processing-n:127.0.0.1:43939_solr) [n:127.0.0.1:43939_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/multicollection2/state.json] for collection [multicollection2] has occurred - updating... (live nodes size: [0])
[junit4] 2> 73308 INFO (zkCallback-86-thread-3-processing-n:127.0.0.1:43939_solr) [n:127.0.0.1:43939_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [0])
[junit4] 2> 73308 INFO (zkCallback-86-thread-1-processing-n:127.0.0.1:43939_solr) [n:127.0.0.1:43939_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/stale_state_test_col/state.json] for collection [stale_state_test_col] has occurred - updating... (live nodes size: [0])
[junit4] 2> 73308 INFO (zkCallback-85-thread-3-processing-n:127.0.0.1:44181_solr) [n:127.0.0.1:44181_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/overwrite/state.json] for collection [overwrite] has occurred - updating... (live nodes size: [0])
[junit4] 2> 73309 INFO (zkCallback-85-thread-1-processing-n:127.0.0.1:44181_solr) [n:127.0.0.1:44181_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/overwrite/state.json] for collection [overwrite] has occurred - updating... (live nodes size: [0])
[junit4] 2> 73310 INFO (jetty-closer-70-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@11b27d8{/solr,null,UNAVAILABLE}
[junit4] 2> 73310 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15fa4ccc2b80007, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 73313 INFO (jetty-closer-70-thread-1) [ ] o.a.s.c.Overseer Overseer (id=98978916172562440-127.0.0.1:44181_solr-n_0000000000) closing
[junit4] 2> 73313 WARN (OverseerAutoScalingTriggerThread-98978916172562440-127.0.0.1:44181_solr-n_0000000000) [n:127.0.0.1:44181_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 73320 INFO (OverseerStateUpdate-98978916172562440-127.0.0.1:44181_solr-n_0000000000) [n:127.0.0.1:44181_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:44181_solr
[junit4] 2> 73321 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15fa4ccc2b80009, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 73322 INFO (jetty-closer-70-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@4a945e{/solr,null,UNAVAILABLE}
[junit4] 2> 73322 INFO (jetty-closer-70-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@20306e{/solr,null,UNAVAILABLE}
[junit4] 2> 73324 ERROR (SUITE-CloudSolrClientTest-seed#[4470578693723F30]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 73324 INFO (SUITE-CloudSolrClientTest-seed#[4470578693723F30]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46709 46709
[junit4] 2> 78509 INFO (Thread-72) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:46709 46709
[junit4] 2> 78510 WARN (Thread-72) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/aliases.json
[junit4] 2> 3 /solr/security.json
[junit4] 2> 3 /solr/configs/conf
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 21 /solr/collections/foo/state.json
[junit4] 2> 21 /solr/collections/localShardsTestColl/state.json
[junit4] 2> 16 /solr/collections/stale_state_test_col/state.json
[junit4] 2> 12 /solr/collections/collection1/state.json
[junit4] 2> 12 /solr/collections/2nd_collection/state.json
[junit4] 2> 12 /solr/collections/multicollection1/state.json
[junit4] 2> 12 /solr/collections/multicollection2/state.json
[junit4] 2> 10 /solr/collections/nemesis/state.json
[junit4] 2> 8 /solr/clusterprops.json
[junit4] 2> 8 /solr/clusterstate.json
[junit4] 2> 6 /solr/collections/overwrite/state.json
[junit4] 2> 2 /solr/collections/localShardsTestColl/leader_elect/shard2/election/98978916172562440-core_node9-n_0000000000
[junit4] 2> 2 /solr/collections/localShardsTestColl/leader_elect/shard3/election/98978916172562441-core_node17-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/live_nodes
[junit4] 2> 8 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.impl.CloudSolrClientTest_4470578693723F30-001
[junit4] 2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=false): {}, locale=ga, timezone=Cuba
[junit4] 2> NOTE: Linux 4.10.0-37-generic i386/Oracle Corporation 1.8.0_144 (32-bit)/cpus=8,threads=1,free=185618232,total=301465600
[junit4] 2> NOTE: All tests run in this JVM: [PowerEvaluatorTest, DocumentAnalysisResponseTest, HyperbolicSineEvaluatorTest, CloudSolrClientBuilderTest, TestHash, TestCollectionStateWatchers, TestJavaBinCodec, ArrayEvaluatorTest, CloudSolrClientTest]
[junit4] Completed [76/146 (1!)] on J2 in 39.63s, 16 tests, 1 error <<< FAILURES!
[...truncated 42406 lines...]
[JENKINS] Lucene-Solr-7.x-Linux (32bit/jdk1.8.0_144) - Build # 784
- Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/784/
Java: 32bit/jdk1.8.0_144 -client -XX:+UseConcMarkSweepGC
3 tests failed.
FAILED: org.apache.solr.cloud.MissingSegmentRecoveryTest.testLeaderRecovery
Error Message:
Error from server at http://127.0.0.1:43205/solr: create the collection time out:180s
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:43205/solr: create the collection time out:180s
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:883)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.MissingSegmentRecoveryTest.setup(MissingSegmentRecoveryTest.java:61)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:968)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.MultiThreadedOCPTest.test
Error Message:
Error from server at http://127.0.0.1:34677/j: Could not fully create collection: collection1
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:34677/j: Could not fully create collection: collection1
at __randomizedtesting.SeedInfo.seed([53F12E1D8D06E901:DBA511C723FA84F9]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:883)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:384)
at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:333)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.client.solrj.impl.CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale
Error Message:
Error from server at http://127.0.0.1:42895/solr/stale_state_test_col_shard1_replica_n1: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 404 </title> </head> <body> <h2>HTTP ERROR: 404</h2> <p>Problem accessing /solr/stale_state_test_col_shard1_replica_n1/update. Reason: <pre> Can not find: /solr/stale_state_test_col_shard1_replica_n1/update</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.20.v20170531</a><hr/> </body> </html>
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:42895/solr/stale_state_test_col_shard1_replica_n1: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 </title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /solr/stale_state_test_col_shard1_replica_n1/update. Reason:
<pre> Can not find: /solr/stale_state_test_col_shard1_replica_n1/update</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.20.v20170531</a><hr/>
</body>
</html>
at __randomizedtesting.SeedInfo.seed([757E157980472907:C14F8D9163AE5F2B]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:559)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1016)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:883)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
at org.apache.solr.client.solrj.impl.CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale(CloudSolrClientTest.java:844)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12006 lines...]
[junit4] Suite: org.apache.solr.cloud.MultiThreadedOCPTest
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_53F12E1D8D06E901-001/init-core-data-001
[junit4] 2> 548399 INFO (SUITE-MultiThreadedOCPTest-seed#[53F12E1D8D06E901]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 548400 INFO (SUITE-MultiThreadedOCPTest-seed#[53F12E1D8D06E901]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 548400 INFO (SUITE-MultiThreadedOCPTest-seed#[53F12E1D8D06E901]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 548400 INFO (SUITE-MultiThreadedOCPTest-seed#[53F12E1D8D06E901]-worker) [ ] o.a.s.BaseDistributedSearchTestCase Setting hostContext system property: /j/
[junit4] 2> 548402 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 548405 INFO (Thread-518) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 548405 INFO (Thread-518) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 548411 ERROR (Thread-518) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 548505 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.ZkTestServer start zk server on port:34115
[junit4] 2> 548539 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
[junit4] 2> 548541 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
[junit4] 2> 548543 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
[junit4] 2> 548545 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
[junit4] 2> 548547 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
[junit4] 2> 548548 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
[junit4] 2> 548549 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
[junit4] 2> 548552 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
[junit4] 2> 548554 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
[junit4] 2> 548555 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
[junit4] 2> 548556 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.AbstractZkTestCase put /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
[junit4] 2> 548557 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.AbstractFullDistribZkTestBase Will use NRT replicas unless explicitly asked otherwise
[junit4] 2> 548672 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 548673 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@d54e{/j,null,AVAILABLE}
[junit4] 2> 548675 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1a090a2{HTTP/1.1,[http/1.1]}{127.0.0.1:34677}
[junit4] 2> 548675 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.e.j.s.Server Started @550363ms
[junit4] 2> 548675 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_53F12E1D8D06E901-001/tempDir-001/control/data, replicaType=NRT, hostContext=/j, hostPort=34677, coreRootDirectory=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_53F12E1D8D06E901-001/control-001/cores}
[junit4] 2> 548676 ERROR (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 548676 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.2.0
[junit4] 2> 548676 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 548676 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 548676 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-10T03:21:23.632Z
[junit4] 2> 548679 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.s.SolrDispatchFilter Loading solr.xml from SolrHome (not found in ZooKeeper)
[junit4] 2> 548679 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_53F12E1D8D06E901-001/control-001/solr.xml
[junit4] 2> 548687 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 548687 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 548689 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@d517b9, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 548698 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:34115/solr
[junit4] 2> 548775 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [n:127.0.0.1:34677_j ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 548775 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [n:127.0.0.1:34677_j ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:34677_j
[junit4] 2> 548776 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [n:127.0.0.1:34677_j ] o.a.s.c.Overseer Overseer (id=98977977687408644-127.0.0.1:34677_j-n_0000000000) starting
[junit4] 2> 548794 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [n:127.0.0.1:34677_j ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:34677_j
[junit4] 2> 548800 INFO (OverseerStateUpdate-98977977687408644-127.0.0.1:34677_j-n_0000000000) [n:127.0.0.1:34677_j ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 548980 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [n:127.0.0.1:34677_j ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@d517b9
[junit4] 2> 548994 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [n:127.0.0.1:34677_j ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@d517b9
[junit4] 2> 548995 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [n:127.0.0.1:34677_j ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@d517b9
[junit4] 2> 548997 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [n:127.0.0.1:34677_j ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_53F12E1D8D06E901-001/control-001/cores
[junit4] 2> 549028 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 549029 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34115/solr ready
[junit4] 2> 549035 INFO (qtp9550590-3743) [n:127.0.0.1:34677_j ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:34677_j&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 549042 INFO (OverseerThreadFactory-1428-thread-1-processing-n:127.0.0.1:34677_j) [n:127.0.0.1:34677_j ] o.a.s.c.CreateCollectionCmd Create collection control_collection
[junit4] 2> 549151 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 549152 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 549258 INFO (zkCallback-973-thread-1-processing-n:127.0.0.1:34677_j) [n:127.0.0.1:34677_j ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 550195 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.2.0
[junit4] 2> 550225 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema [control_collection_shard1_replica_n1] Schema name=test
[junit4] 2> 550391 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema test/1.0 with uniqueid field id
[junit4] 2> 550422 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'control_collection_shard1_replica_n1' using configuration from collection control_collection, trusted=true
[junit4] 2> 550422 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.control_collection.shard1.replica_n1' (registry 'solr.core.control_collection.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@d517b9
[junit4] 2> 550422 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 550423 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [[control_collection_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_53F12E1D8D06E901-001/control-001/cores/control_collection_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_53F12E1D8D06E901-001/control-001/cores/control_collection_shard1_replica_n1/data/]
[junit4] 2> 550427 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=45, maxMergeAtOnceExplicit=27, maxMergedSegmentMB=68.9736328125, floorSegmentMB=0.353515625, forceMergeDeletesPctAllowed=11.17814230767723, segmentsPerTier=15.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.3295227805276786
[junit4] 2> 550430 WARN (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.RequestHandlers INVALID paramSet a in requestHandler {type = requestHandler,name = /dump,class = DumpRequestHandler,attributes = {initParams=a, name=/dump, class=DumpRequestHandler},args = {defaults={a=A,b=B}}}
[junit4] 2> 550494 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 550494 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 550496 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 550496 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 550498 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=47, maxMergeAtOnceExplicit=11, maxMergedSegmentMB=62.220703125, floorSegmentMB=1.09765625, forceMergeDeletesPctAllowed=28.933221561459877, segmentsPerTier=38.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4821812671409099
[junit4] 2> 550499 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1a9dd8f[control_collection_shard1_replica_n1] main]
[junit4] 2> 550500 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
[junit4] 2> 550502 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
[junit4] 2> 550503 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 550505 INFO (searcherExecutor-1431-thread-1-processing-n:127.0.0.1:34677_j x:control_collection_shard1_replica_n1 s:shard1 c:control_collection) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] Registered new searcher Searcher@1a9dd8f[control_collection_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 550505 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1583647645196353536
[junit4] 2> 550509 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 550509 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 550509 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:34677/j/control_collection_shard1_replica_n1/
[junit4] 2> 550509 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 550509 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:34677/j/control_collection_shard1_replica_n1/ has no replicas
[junit4] 2> 550509 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 550511 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:34677/j/control_collection_shard1_replica_n1/ shard1
[junit4] 2> 550613 INFO (zkCallback-973-thread-2-processing-n:127.0.0.1:34677_j) [n:127.0.0.1:34677_j ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/control_collection/state.json] for collection [control_collection] has occurred - updating... (live nodes size: [1])
[junit4] 2> 550662 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 550670 INFO (qtp9550590-3740) [n:127.0.0.1:34677_j c:control_collection s:shard1 x:control_collection_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=conf1&newCollection=true&name=control_collection_shard1_replica_n1&action=CREATE&numShards=1&collection=control_collection&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1519
[junit4] 2> 550674 INFO (qtp9550590-3743) [n:127.0.0.1:34677_j ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 551048 INFO (OverseerCollectionConfigSetProcessor-98977977687408644-127.0.0.1:34677_j-n_0000000000) [n:127.0.0.1:34677_j ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 580677 ERROR (qtp9550590-3743) [n:127.0.0.1:34677_j ] o.a.s.h.a.CollectionsHandler Timed out waiting for new collection's replicas to become ACTIVE replica http://127.0.0.1:34677/j/control_collection_shard1_replica_n1/ is in state of down with timeout=30
[junit4] 2> 580678 INFO (qtp9550590-3743) [n:127.0.0.1:34677_j ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=control_collection&nrtReplicas=1&action=CREATE&numShards=1&createNodeSet=127.0.0.1:34677_j&wt=javabin&version=2} status=0 QTime=31642
[junit4] 2> 580687 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 580687 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:34115/solr ready
[junit4] 2> 580687 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.ChaosMonkey monkey: init - expire sessions:false cause connection loss:false
[junit4] 2> 580689 INFO (qtp9550590-3743) [n:127.0.0.1:34677_j ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 580692 INFO (OverseerThreadFactory-1428-thread-2-processing-n:127.0.0.1:34677_j) [n:127.0.0.1:34677_j ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 580693 WARN (OverseerThreadFactory-1428-thread-2-processing-n:127.0.0.1:34677_j) [n:127.0.0.1:34677_j ] o.a.s.c.CreateCollectionCmd It is unusual to create a collection (collection1) without cores.
[junit4] 2> 610735 ERROR (OverseerThreadFactory-1428-thread-2-processing-n:127.0.0.1:34677_j) [n:127.0.0.1:34677_j ] o.a.s.c.OverseerCollectionMessageHandler Collection: collection1 operation: create failed:org.apache.solr.common.SolrException: Could not fully create collection: collection1
[junit4] 2> at org.apache.solr.cloud.CreateCollectionCmd.call(CreateCollectionCmd.java:215)
[junit4] 2> at org.apache.solr.cloud.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:232)
[junit4] 2> at org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:464)
[junit4] 2> at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 610743 INFO (qtp9550590-3743) [n:127.0.0.1:34677_j ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=conf1&name=collection1&nrtReplicas=1&action=CREATE&numShards=2&createNodeSet=&stateFormat=2&wt=javabin&version=2} status=500 QTime=30053
[junit4] 2> 610743 ERROR (qtp9550590-3743) [n:127.0.0.1:34677_j ] o.a.s.s.HttpSolrCall null:org.apache.solr.common.SolrException: Could not fully create collection: collection1
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:307)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:244)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:222)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:737)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:718)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:499)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 610745 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.ChaosMonkey monkey: stop jetty! 34677
[junit4] 2> 610745 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=15704704
[junit4] 2> 610745 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 610745 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@919153: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@1f3cde2
[junit4] 2> 610751 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 610751 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@13493cc: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@1fed8d
[junit4] 2> 610756 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 610756 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5ef334: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@c4d3c0
[junit4] 2> 610757 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:34677_j
[junit4] 2> 610758 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 610758 INFO (zkCallback-973-thread-1-processing-n:127.0.0.1:34677_j) [n:127.0.0.1:34677_j ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 610759 INFO (coreCloseExecutor-1436-thread-1) [n:127.0.0.1:34677_j c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.c.SolrCore [control_collection_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@1a0b5f
[junit4] 2> 610759 INFO (coreCloseExecutor-1436-thread-1) [n:127.0.0.1:34677_j c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.control_collection.shard1.replica_n1, tag=1706847
[junit4] 2> 610759 INFO (coreCloseExecutor-1436-thread-1) [n:127.0.0.1:34677_j c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@d575fe: rootName = null, domain = solr.core.control_collection.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.control_collection.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@17da287
[junit4] 2> 610773 INFO (coreCloseExecutor-1436-thread-1) [n:127.0.0.1:34677_j c:control_collection s:shard1 r:core_node2 x:control_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.control_collection.shard1.leader, tag=1706847
[junit4] 2> 610775 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.Overseer Overseer (id=98977977687408644-127.0.0.1:34677_j-n_0000000000) closing
[junit4] 2> 610775 INFO (OverseerStateUpdate-98977977687408644-127.0.0.1:34677_j-n_0000000000) [n:127.0.0.1:34677_j ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:34677_j
[junit4] 2> 610775 WARN (OverseerAutoScalingTriggerThread-98977977687408644-127.0.0.1:34677_j-n_0000000000) [n:127.0.0.1:34677_j ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 610778 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@1a090a2{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 610778 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@d54e{/j,null,UNAVAILABLE}
[junit4] 2> 610779 ERROR (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 610779 INFO (TEST-MultiThreadedOCPTest.test-seed#[53F12E1D8D06E901]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34115 34115
[junit4] 2> 612981 INFO (Thread-518) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:34115 34115
[junit4] 2> 612981 WARN (Thread-518) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/live_nodes
[junit4] 2> 3 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=MultiThreadedOCPTest -Dtests.method=test -Dtests.seed=53F12E1D8D06E901 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=nl-NL -Dtests.timezone=America/Vancouver -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
[junit4] ERROR 64.6s J1 | MultiThreadedOCPTest.test <<<
[junit4] > Throwable #1: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:34677/j: Could not fully create collection: collection1
[junit4] > at __randomizedtesting.SeedInfo.seed([53F12E1D8D06E901:DBA511C723FA84F9]:0)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:643)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
[junit4] > at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
[junit4] > at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1104)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:883)
[junit4] > at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
[junit4] > at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:211)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createJettys(AbstractFullDistribZkTestBase.java:384)
[junit4] > at org.apache.solr.cloud.AbstractFullDistribZkTestBase.createServers(AbstractFullDistribZkTestBase.java:333)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsFixedStatement.callStatement(BaseDistributedSearchTestCase.java:991)
[junit4] > at org.apache.solr.BaseDistributedSearchTestCase$ShardsRepeatRule$ShardsStatement.evaluate(BaseDistributedSearchTestCase.java:968)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MultiThreadedOCPTest_53F12E1D8D06E901-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {}, docValues:{}, maxPointsInLeafNode=1268, maxMBSortInHeap=7.557761437925071, sim=RandomSimilarity(queryNorm=true): {}, locale=nl-NL, timezone=America/Vancouver
[junit4] 2> NOTE: Linux 4.10.0-37-generic i386/Oracle Corporation 1.8.0_144 (32-bit)/cpus=8,threads=1,free=73546008,total=179879936
[junit4] 2> NOTE: All tests run in this JVM: [TestCloudNestedDocsSort, SolrGraphiteReporterTest, TestOmitPositions, CloudExitableDirectoryReaderTest, TestFaceting, TestDocTermOrds, RuleEngineTest, TestReplicaProperties, TestMacros, TestDocumentBuilder, TestDynamicLoading, TestLuceneMatchVersion, TestSolr4Spatial, OutputWriterTest, TestConfigSetsAPIExclusivity, TestExpandComponent, TestSolrCloudWithKerberosAlt, BadIndexSchemaTest, TestCollationField, TestPseudoReturnFields, BitVectorTest, TestStressLucene, TestExclusionRuleCollectionAccess, SignatureUpdateProcessorFactoryTest, TestSchemaResource, PreAnalyzedFieldTest, SuggesterFSTTest, TestUseDocValuesAsStored, WrapperMergePolicyFactoryTest, BufferStoreTest, ConvertedLegacyTest, SpellingQueryConverterTest, PathHierarchyTokenizerFactoryTest, ShardRoutingCustomTest, TestRetrieveFieldsOptimizer, DOMUtilTest, TestCollectionAPI, ZkCLITest, LukeRequestHandlerTest, SpatialRPTFieldTypeTest, SuggestComponentContextFilterQueryTest, SearchRateTriggerTest, BasicAuthStandaloneTest, RequestHandlersTest, TestUnifiedSolrHighlighter, TestRTGBase, TestOverriddenPrefixQueryForCustomFieldType, TestSizeLimitedDistributedMap, TestWriterPerf, ReplaceNodeTest, TestAddFieldRealTimeGet, RequestLoggingTest, TestPushWriter, MultiThreadedOCPTest]
[junit4] Completed [148/748 (1!)] on J1 in 64.66s, 1 test, 1 error <<< FAILURES!
[...truncated 1085 lines...]
[junit4] Suite: org.apache.solr.cloud.MissingSegmentRecoveryTest
[junit4] 2> 2110341 INFO (SUITE-MissingSegmentRecoveryTest-seed#[53F12E1D8D06E901]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_53F12E1D8D06E901-001/init-core-data-001
[junit4] 2> 2110342 WARN (SUITE-MissingSegmentRecoveryTest-seed#[53F12E1D8D06E901]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=32 numCloses=32
[junit4] 2> 2110342 INFO (SUITE-MissingSegmentRecoveryTest-seed#[53F12E1D8D06E901]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 2110343 INFO (SUITE-MissingSegmentRecoveryTest-seed#[53F12E1D8D06E901]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, ssl=NaN, value=NaN, clientAuth=NaN)
[junit4] 2> 2110344 INFO (SUITE-MissingSegmentRecoveryTest-seed#[53F12E1D8D06E901]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_53F12E1D8D06E901-001/tempDir-001
[junit4] 2> 2110344 INFO (SUITE-MissingSegmentRecoveryTest-seed#[53F12E1D8D06E901]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2110344 INFO (Thread-3716) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2110344 INFO (Thread-3716) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2110346 ERROR (Thread-3716) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2110444 INFO (SUITE-MissingSegmentRecoveryTest-seed#[53F12E1D8D06E901]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:43817
[junit4] 2> 2110454 INFO (jetty-launcher-2595-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2110455 INFO (jetty-launcher-2595-thread-2) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2110455 INFO (jetty-launcher-2595-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@5441b{/solr,null,AVAILABLE}
[junit4] 2> 2110455 INFO (jetty-launcher-2595-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3b03f0{/solr,null,AVAILABLE}
[junit4] 2> 2110459 INFO (jetty-launcher-2595-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@180fcc{HTTP/1.1,[http/1.1]}{127.0.0.1:43205}
[junit4] 2> 2110459 INFO (jetty-launcher-2595-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1005691{HTTP/1.1,[http/1.1]}{127.0.0.1:45631}
[junit4] 2> 2110459 INFO (jetty-launcher-2595-thread-2) [ ] o.e.j.s.Server Started @2112147ms
[junit4] 2> 2110459 INFO (jetty-launcher-2595-thread-1) [ ] o.e.j.s.Server Started @2112147ms
[junit4] 2> 2110459 INFO (jetty-launcher-2595-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=43205}
[junit4] 2> 2110459 INFO (jetty-launcher-2595-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=45631}
[junit4] 2> 2110459 ERROR (jetty-launcher-2595-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2110459 ERROR (jetty-launcher-2595-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2110459 INFO (jetty-launcher-2595-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.2.0
[junit4] 2> 2110459 INFO (jetty-launcher-2595-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.2.0
[junit4] 2> 2110459 INFO (jetty-launcher-2595-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2110459 INFO (jetty-launcher-2595-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2110459 INFO (jetty-launcher-2595-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2110459 INFO (jetty-launcher-2595-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2110459 INFO (jetty-launcher-2595-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-10T03:47:25.415Z
[junit4] 2> 2110459 INFO (jetty-launcher-2595-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-10T03:47:25.415Z
[junit4] 2> 2110462 INFO (jetty-launcher-2595-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2110462 INFO (jetty-launcher-2595-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2110470 INFO (jetty-launcher-2595-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43817/solr
[junit4] 2> 2110470 INFO (jetty-launcher-2595-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:43817/solr
[junit4] 2> 2110472 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15fa40a15f50003, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2110516 INFO (jetty-launcher-2595-thread-1) [n:127.0.0.1:45631_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2110517 INFO (jetty-launcher-2595-thread-1) [n:127.0.0.1:45631_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:45631_solr
[junit4] 2> 2110517 INFO (jetty-launcher-2595-thread-1) [n:127.0.0.1:45631_solr ] o.a.s.c.Overseer Overseer (id=98978080050446341-127.0.0.1:45631_solr-n_0000000000) starting
[junit4] 2> 2110518 INFO (jetty-launcher-2595-thread-2) [n:127.0.0.1:43205_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2110519 INFO (jetty-launcher-2595-thread-2) [n:127.0.0.1:43205_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:43205_solr
[junit4] 2> 2110520 INFO (zkCallback-2606-thread-1-processing-n:127.0.0.1:45631_solr) [n:127.0.0.1:45631_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2110522 INFO (zkCallback-2607-thread-1-processing-n:127.0.0.1:43205_solr) [n:127.0.0.1:43205_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2110528 INFO (jetty-launcher-2595-thread-1) [n:127.0.0.1:45631_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:45631_solr
[junit4] 2> 2110533 INFO (zkCallback-2606-thread-1-processing-n:127.0.0.1:45631_solr) [n:127.0.0.1:45631_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2110533 INFO (zkCallback-2607-thread-1-processing-n:127.0.0.1:43205_solr) [n:127.0.0.1:43205_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2110693 INFO (jetty-launcher-2595-thread-2) [n:127.0.0.1:43205_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43205.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@d517b9
[junit4] 2> 2110702 INFO (jetty-launcher-2595-thread-2) [n:127.0.0.1:43205_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43205.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@d517b9
[junit4] 2> 2110702 INFO (jetty-launcher-2595-thread-2) [n:127.0.0.1:43205_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43205.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@d517b9
[junit4] 2> 2110704 INFO (jetty-launcher-2595-thread-2) [n:127.0.0.1:43205_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_53F12E1D8D06E901-001/tempDir-001/node2/.
[junit4] 2> 2110757 INFO (jetty-launcher-2595-thread-1) [n:127.0.0.1:45631_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45631.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@d517b9
[junit4] 2> 2110765 INFO (jetty-launcher-2595-thread-1) [n:127.0.0.1:45631_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45631.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@d517b9
[junit4] 2> 2110765 INFO (jetty-launcher-2595-thread-1) [n:127.0.0.1:45631_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45631.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@d517b9
[junit4] 2> 2110766 INFO (jetty-launcher-2595-thread-1) [n:127.0.0.1:45631_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_53F12E1D8D06E901-001/tempDir-001/node1/.
[junit4] 2> 2110792 INFO (SUITE-MissingSegmentRecoveryTest-seed#[53F12E1D8D06E901]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2110792 INFO (SUITE-MissingSegmentRecoveryTest-seed#[53F12E1D8D06E901]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:43817/solr ready
[junit4] 2> 2110862 INFO (TEST-MissingSegmentRecoveryTest.testLeaderRecovery-seed#[53F12E1D8D06E901]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testLeaderRecovery
[junit4] 2> 2110863 INFO (qtp7254915-15920) [n:127.0.0.1:43205_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=2&collection.configName=conf&maxShardsPerNode=1&name=MissingSegmentRecoveryTest&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2110866 INFO (OverseerThreadFactory-5924-thread-1-processing-n:127.0.0.1:45631_solr) [n:127.0.0.1:45631_solr ] o.a.s.c.CreateCollectionCmd Create collection MissingSegmentRecoveryTest
[junit4] 2> 2110976 INFO (OverseerStateUpdate-98978080050446341-127.0.0.1:45631_solr-n_0000000000) [n:127.0.0.1:45631_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"MissingSegmentRecoveryTest",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"MissingSegmentRecoveryTest_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:45631/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2110979 INFO (OverseerStateUpdate-98978080050446341-127.0.0.1:45631_solr-n_0000000000) [n:127.0.0.1:45631_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"MissingSegmentRecoveryTest",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"MissingSegmentRecoveryTest_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:43205/solr",
[junit4] 2> "type":"NRT",
[junit4] 2> "waitForFinalState":"false"}
[junit4] 2> 2111182 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=MissingSegmentRecoveryTest_shard1_replica_n2&action=CREATE&numShards=1&collection=MissingSegmentRecoveryTest&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2111182 INFO (qtp16738099-15910) [n:127.0.0.1:45631_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=MissingSegmentRecoveryTest_shard1_replica_n1&action=CREATE&numShards=1&collection=MissingSegmentRecoveryTest&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2111183 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2111183 INFO (qtp16738099-15910) [n:127.0.0.1:45631_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2111289 INFO (zkCallback-2606-thread-1-processing-n:127.0.0.1:45631_solr) [n:127.0.0.1:45631_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2111289 INFO (zkCallback-2607-thread-1-processing-n:127.0.0.1:43205_solr) [n:127.0.0.1:43205_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2111289 INFO (zkCallback-2607-thread-2-processing-n:127.0.0.1:43205_solr) [n:127.0.0.1:43205_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2111290 INFO (zkCallback-2606-thread-2-processing-n:127.0.0.1:45631_solr) [n:127.0.0.1:45631_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/MissingSegmentRecoveryTest/state.json] for collection [MissingSegmentRecoveryTest] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2112202 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.2.0
[junit4] 2> 2112202 INFO (qtp16738099-15910) [n:127.0.0.1:45631_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node3 x:MissingSegmentRecoveryTest_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.2.0
[junit4] 2> 2112239 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.s.IndexSchema [MissingSegmentRecoveryTest_shard1_replica_n2] Schema name=minimal
[junit4] 2> 2112239 INFO (qtp16738099-15910) [n:127.0.0.1:45631_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node3 x:MissingSegmentRecoveryTest_shard1_replica_n1] o.a.s.s.IndexSchema [MissingSegmentRecoveryTest_shard1_replica_n1] Schema name=minimal
[junit4] 2> 2112241 INFO (qtp16738099-15910) [n:127.0.0.1:45631_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node3 x:MissingSegmentRecoveryTest_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2112241 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2112241 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'MissingSegmentRecoveryTest_shard1_replica_n2' using configuration from collection MissingSegmentRecoveryTest, trusted=true
[junit4] 2> 2112241 INFO (qtp16738099-15910) [n:127.0.0.1:45631_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node3 x:MissingSegmentRecoveryTest_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'MissingSegmentRecoveryTest_shard1_replica_n1' using configuration from collection MissingSegmentRecoveryTest, trusted=true
[junit4] 2> 2112242 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_43205.solr.core.MissingSegmentRecoveryTest.shard1.replica_n2' (registry 'solr.core.MissingSegmentRecoveryTest.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@d517b9
[junit4] 2> 2112242 INFO (qtp16738099-15910) [n:127.0.0.1:45631_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node3 x:MissingSegmentRecoveryTest_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_45631.solr.core.MissingSegmentRecoveryTest.shard1.replica_n1' (registry 'solr.core.MissingSegmentRecoveryTest.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@d517b9
[junit4] 2> 2112242 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2112242 INFO (qtp16738099-15910) [n:127.0.0.1:45631_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node3 x:MissingSegmentRecoveryTest_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2112242 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.c.SolrCore [[MissingSegmentRecoveryTest_shard1_replica_n2] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_53F12E1D8D06E901-001/tempDir-001/node2/MissingSegmentRecoveryTest_shard1_replica_n2], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_53F12E1D8D06E901-001/tempDir-001/node2/./MissingSegmentRecoveryTest_shard1_replica_n2/data/]
[junit4] 2> 2112242 INFO (qtp16738099-15910) [n:127.0.0.1:45631_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node3 x:MissingSegmentRecoveryTest_shard1_replica_n1] o.a.s.c.SolrCore [[MissingSegmentRecoveryTest_shard1_replica_n1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_53F12E1D8D06E901-001/tempDir-001/node1/MissingSegmentRecoveryTest_shard1_replica_n1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J1/temp/solr.cloud.MissingSegmentRecoveryTest_53F12E1D8D06E901-001/tempDir-001/node1/./MissingSegmentRecoveryTest_shard1_replica_n1/data/]
[junit4] 2> 2112607 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2112607 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2112608 INFO (qtp16738099-15910) [n:127.0.0.1:45631_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node3 x:MissingSegmentRecoveryTest_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2112608 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2112608 INFO (qtp16738099-15910) [n:127.0.0.1:45631_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node3 x:MissingSegmentRecoveryTest_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2112609 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2112610 INFO (qtp16738099-15910) [n:127.0.0.1:45631_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node3 x:MissingSegmentRecoveryTest_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2112610 INFO (qtp16738099-15910) [n:127.0.0.1:45631_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node3 x:MissingSegmentRecoveryTest_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2112611 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@3e74d2[MissingSegmentRecoveryTest_shard1_replica_n2] main]
[junit4] 2> 2112611 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2112612 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2112612 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2112613 INFO (qtp16738099-15910) [n:127.0.0.1:45631_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node3 x:MissingSegmentRecoveryTest_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@10d0157[MissingSegmentRecoveryTest_shard1_replica_n1] main]
[junit4] 2> 2112613 INFO (qtp16738099-15910) [n:127.0.0.1:45631_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node3 x:MissingSegmentRecoveryTest_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
[junit4] 2> 2112613 INFO (searcherExecutor-5929-thread-1-processing-n:127.0.0.1:43205_solr x:MissingSegmentRecoveryTest_shard1_replica_n2 s:shard1 c:MissingSegmentRecoveryTest r:core_node4) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.c.SolrCore [MissingSegmentRecoveryTest_shard1_replica_n2] Registered new searcher Searcher@3e74d2[MissingSegmentRecoveryTest_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2112613 INFO (qtp16738099-15910) [n:127.0.0.1:45631_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node3 x:MissingSegmentRecoveryTest_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
[junit4] 2> 2112613 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1583649283185311744
[junit4] 2> 2112614 INFO (qtp16738099-15910) [n:127.0.0.1:45631_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node3 x:MissingSegmentRecoveryTest_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2112616 INFO (searcherExecutor-5930-thread-1-processing-n:127.0.0.1:45631_solr x:MissingSegmentRecoveryTest_shard1_replica_n1 s:shard1 c:MissingSegmentRecoveryTest r:core_node3) [n:127.0.0.1:45631_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node3 x:MissingSegmentRecoveryTest_shard1_replica_n1] o.a.s.c.SolrCore [MissingSegmentRecoveryTest_shard1_replica_n1] Registered new searcher Searcher@10d0157[MissingSegmentRecoveryTest_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2112616 INFO (qtp16738099-15910) [n:127.0.0.1:45631_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node3 x:MissingSegmentRecoveryTest_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1583649283188457472
[junit4] 2> 2112617 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 2113118 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2113118 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2113118 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:43205/solr/MissingSegmentRecoveryTest_shard1_replica_n2/
[junit4] 2> 2113119 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=MissingSegmentRecoveryTest_shard1_replica_n2 url=http://127.0.0.1:43205/solr START replicas=[http://127.0.0.1:45631/solr/MissingSegmentRecoveryTest_shard1_replica_n1/] nUpdates=100
[junit4] 2> 2113125 INFO (qtp16738099-15912) [n:127.0.0.1:45631_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node3 x:MissingSegmentRecoveryTest_shard1_replica_n1] o.a.s.c.S.Request [MissingSegmentRecoveryTest_shard1_replica_n1] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=3
[junit4] 2> 2113420 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.u.PeerSync PeerSync: core=MissingSegmentRecoveryTest_shard1_replica_n2 url=http://127.0.0.1:43205/solr DONE. We have no versions. sync failed.
[junit4] 2> 2113420 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 2113420 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 2113420 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2113425 INFO (qtp7254915-15916) [n:127.0.0.1:43205_solr c:MissingSegmentRecoveryTest s:shard1 r:core_node4 x:MissingSegmentRecoveryTest_shard1_replica_n2] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:43205/solr/MissingSegmentRecoveryTest_shard1_replica_n2/ shard1
[junit4] 2> 2290868 ERROR (qtp7254915-15920) [n:127.0.0.1:43205_solr ] o.a.s.h.RequestHandlerBase org.apache.solr.common.SolrException: create the collection time out:180s
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:312)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:244)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:222)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:737)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:718)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:499)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4] 2> at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:283)
[junit4] 2> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:108)
[junit4] 2> at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
[junit4] 2> at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
[junit4] 2> at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2>
[junit4] 2> 2290874 INFO (qtp7254915-15920) [n:127.0.0.1:43205_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=2&collection.configName=conf&maxShardsPerNode=1&name=MissingSegmentRecoveryTest&nrtReplicas=2&action=CREATE&numShards=1&wt=javabin&version=2} status=500 QTime=180010
[junit4] 2> 2290875 ERROR (qtp7254915-15920) [n:127.0.0.1:43205_solr ] o.a.s.s.HttpSolrCall null:org.apache.solr.common.SolrException: create the collection time out:180s
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleResponse(CollectionsHandler.java:312)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.invokeAction(CollectionsHandler.java:244)
[junit4] 2> at org.apache.solr.handler.admin.CollectionsHandler.handleRequestBody(CollectionsHandler.java:222)
[junit4] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:177)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdmin(HttpSolrCall.java:737)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:718)
[junit4] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:499)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:382)
[junit4] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:326)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1759)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:224)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
[junit4] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
[junit4] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
[junit4] 2> at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
[junit4] 2> at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
[junit4] 2> at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:493)
[junit4] 2> at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
[junit4] 2> at org.eclipse.jetty.server.Server.handle(Server.java:534)
[junit4] 2> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
[junit4] 2> at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
[junit4
[...truncated too long message...]
client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 138852 INFO (jetty-closer-188-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@113c18{/solr,null,UNAVAILABLE}
[junit4] 2> 138856 INFO (coreCloseExecutor-746-thread-7) [n:127.0.0.1:36751_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n16] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.localShardsTestColl.shard3.replica_n16, tag=4398627
[junit4] 2> 138856 INFO (coreCloseExecutor-746-thread-7) [n:127.0.0.1:36751_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n16] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1a0d843: rootName = solr_36751, domain = solr.core.localShardsTestColl.shard3.replica_n16, service url = null, agent id = null] for registry solr.core.localShardsTestColl.shard3.replica_n16 / com.codahale.metrics.MetricRegistry@14f1246
[junit4] 2> 138863 INFO (coreCloseExecutor-747-thread-7) [n:127.0.0.1:41553_solr c:localShardsTestColl s:shard1 r:core_node3 x:localShardsTestColl_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard1.leader, tag=27649200
[junit4] 2> 138863 INFO (coreCloseExecutor-747-thread-1) [n:127.0.0.1:41553_solr c:multicollection2 s:shard1 r:core_node3 x:multicollection2_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.multicollection2.shard1.leader, tag=17350278
[junit4] 2> 138863 INFO (coreCloseExecutor-747-thread-2) [n:127.0.0.1:41553_solr c:2nd_collection s:shard2 r:core_node4 x:2nd_collection_shard2_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.2nd_collection.shard2.leader, tag=6095371
[junit4] 2> 138864 INFO (coreCloseExecutor-747-thread-5) [n:127.0.0.1:41553_solr c:localShardsTestColl s:shard2 r:core_node9 x:localShardsTestColl_shard2_replica_n6] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard2.leader, tag=4235987
[junit4] 2> 138864 INFO (coreCloseExecutor-747-thread-6) [n:127.0.0.1:41553_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n12] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard3.leader, tag=26312716
[junit4] 2> 138871 INFO (jetty-closer-188-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@15419ae{/solr,null,UNAVAILABLE}
[junit4] 2> 138873 INFO (coreCloseExecutor-746-thread-9) [n:127.0.0.1:36751_solr c:localShardsTestColl s:shard2 r:core_node13 x:localShardsTestColl_shard2_replica_n10] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.localShardsTestColl.shard2.replica_n10, tag=29438744
[junit4] 2> 138873 INFO (coreCloseExecutor-746-thread-9) [n:127.0.0.1:36751_solr c:localShardsTestColl s:shard2 r:core_node13 x:localShardsTestColl_shard2_replica_n10] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@ed919d: rootName = solr_36751, domain = solr.core.localShardsTestColl.shard2.replica_n10, service url = null, agent id = null] for registry solr.core.localShardsTestColl.shard2.replica_n10 / com.codahale.metrics.MetricRegistry@9b911
[junit4] 2> 138886 INFO (coreCloseExecutor-746-thread-9) [n:127.0.0.1:36751_solr c:localShardsTestColl s:shard2 r:core_node13 x:localShardsTestColl_shard2_replica_n10] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard2.leader, tag=29438744
[junit4] 2> 138886 INFO (coreCloseExecutor-746-thread-2) [n:127.0.0.1:36751_solr c:multicollection2 s:shard2 r:core_node4 x:multicollection2_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.multicollection2.shard2.leader, tag=15875261
[junit4] 2> 138887 INFO (coreCloseExecutor-746-thread-1) [n:127.0.0.1:36751_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard2.leader, tag=18252641
[junit4] 2> 138887 INFO (coreCloseExecutor-746-thread-5) [n:127.0.0.1:36751_solr c:2nd_collection s:shard1 r:core_node2 x:2nd_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.2nd_collection.shard1.leader, tag=17822354
[junit4] 2> 138887 INFO (coreCloseExecutor-746-thread-4) [n:127.0.0.1:36751_solr c:stale_state_test_col s:shard1 r:core_node4 x:stale_state_test_col_shard1_replica_n3] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.stale_state_test_col.shard1.leader, tag=30205979
[junit4] 2> 138887 INFO (coreCloseExecutor-746-thread-6) [n:127.0.0.1:36751_solr c:foo s:shard1 r:core_node6 x:foo_shard1_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.foo.shard1.leader, tag=17157426
[junit4] 2> 138888 INFO (coreCloseExecutor-746-thread-8) [n:127.0.0.1:36751_solr c:localShardsTestColl s:shard1 r:core_node7 x:localShardsTestColl_shard1_replica_n4] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard1.leader, tag=18393920
[junit4] 2> 138888 INFO (coreCloseExecutor-746-thread-7) [n:127.0.0.1:36751_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n16] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard3.leader, tag=4398627
[junit4] 2> 138891 INFO (jetty-closer-188-thread-2) [ ] o.a.s.c.Overseer Overseer (id=98978197614493701-127.0.0.1:36751_solr-n_0000000000) closing
[junit4] 2> 138891 INFO (OverseerStateUpdate-98978197614493701-127.0.0.1:36751_solr-n_0000000000) [n:127.0.0.1:36751_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:36751_solr
[junit4] 2> 138892 WARN (OverseerAutoScalingTriggerThread-98978197614493701-127.0.0.1:36751_solr-n_0000000000) [n:127.0.0.1:36751_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 138893 INFO (jetty-closer-188-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1addba1{/solr,null,UNAVAILABLE}
[junit4] 2> 138893 ERROR (SUITE-CloudSolrClientTest-seed#[757E157980472907]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 138893 INFO (SUITE-CloudSolrClientTest-seed#[757E157980472907]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:42225 42225
[junit4] 2> 143932 INFO (Thread-187) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:42225 42225
[junit4] 2> 143933 WARN (Thread-187) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/clusterprops.json
[junit4] 2> 8 /solr/aliases.json
[junit4] 2> 3 /solr/security.json
[junit4] 2> 3 /solr/configs/conf
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 43 /solr/collections/localShardsTestColl/state.json
[junit4] 2> 24 /solr/collections/foo/state.json
[junit4] 2> 16 /solr/collections/stale_state_test_col/state.json
[junit4] 2> 16 /solr/collections/multicollection2/state.json
[junit4] 2> 16 /solr/collections/multicollection1/state.json
[junit4] 2> 14 /solr/collections/collection1/state.json
[junit4] 2> 14 /solr/collections/2nd_collection/state.json
[junit4] 2> 10 /solr/collections/nemesis/state.json
[junit4] 2> 8 /solr/clusterstate.json
[junit4] 2> 6 /solr/collections/overwrite/state.json
[junit4] 2> 2 /solr/overseer_elect/election/98978197614493701-127.0.0.1:36751_solr-n_0000000000
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/live_nodes
[junit4] 2> 8 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.impl.CloudSolrClientTest_757E157980472907-001
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=true): {}, locale=sq, timezone=CST6CDT
[junit4] 2> NOTE: Linux 4.10.0-37-generic i386/Oracle Corporation 1.8.0_144 (32-bit)/cpus=8,threads=1,free=74796040,total=179933184
[junit4] 2> NOTE: All tests run in this JVM: [TestHash, UuidEvaluatorTest, OperationsTest, TestCollectionStateWatchers, DocumentAnalysisResponseTest, GreaterThanEqualToEvaluatorTest, CloudSolrClientBuilderTest, DivideEvaluatorTest, SolrExceptionTest, NoOpResponseParserTest, CosineEvaluatorTest, TestCoreAdmin, RegressionEvaluatorTest, GetByIdTest, MultiplyEvaluatorTest, TestCloudSolrClientConnections, AddEvaluatorTest, ExclusiveOrEvaluatorTest, TestJsonRecordReader, SelectWithEvaluatorsTest, NamedListTest, SolrExampleJettyTest, PowerEvaluatorTest, JettyWebappTest, SolrParamTest, TestEmbeddedSolrServer, TestNamedListCodec, TestDocumentObjectBinder, ModifiableSolrParamsTest, AnlysisResponseBaseTest, UsingSolrJRefGuideExamplesTest, TestPolicy, CloudSolrClientMultiConstructorTest, CloudSolrClientTest]
[junit4] Completed [108/146 (1!)] on J2 in 41.86s, 16 tests, 1 error <<< FAILURES!
[...truncated 42294 lines...]
[JENKINS] Lucene-Solr-7.x-Linux (32bit/jdk1.8.0_144) - Build # 783
- Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Linux/783/
Java: 32bit/jdk1.8.0_144 -server -XX:+UseG1GC
3 tests failed.
FAILED: org.apache.solr.client.solrj.impl.CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale
Error Message:
Error from server at https://127.0.0.1:38643/solr/stale_state_test_col_shard1_replica_n1: Expected mime type application/octet-stream but got text/html. <html> <head> <meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/> <title>Error 404 </title> </head> <body> <h2>HTTP ERROR: 404</h2> <p>Problem accessing /solr/stale_state_test_col_shard1_replica_n1/update. Reason: <pre> Can not find: /solr/stale_state_test_col_shard1_replica_n1/update</pre></p> <hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.20.v20170531</a><hr/> </body> </html>
Stack Trace:
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://127.0.0.1:38643/solr/stale_state_test_col_shard1_replica_n1: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=ISO-8859-1"/>
<title>Error 404 </title>
</head>
<body>
<h2>HTTP ERROR: 404</h2>
<p>Problem accessing /solr/stale_state_test_col_shard1_replica_n1/update. Reason:
<pre> Can not find: /solr/stale_state_test_col_shard1_replica_n1/update</pre></p>
<hr /><a href="http://eclipse.org/jetty">Powered by Jetty:// 9.3.20.v20170531</a><hr/>
</body>
</html>
at __randomizedtesting.SeedInfo.seed([1D670CA03ECF9456:A9569448DD26E27A]:0)
at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:607)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:255)
at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:244)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:483)
at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:413)
at org.apache.solr.client.solrj.impl.CloudSolrClient.directUpdate(CloudSolrClient.java:559)
at org.apache.solr.client.solrj.impl.CloudSolrClient.sendRequest(CloudSolrClient.java:1016)
at org.apache.solr.client.solrj.impl.CloudSolrClient.requestWithRetryOnStaleState(CloudSolrClient.java:883)
at org.apache.solr.client.solrj.impl.CloudSolrClient.request(CloudSolrClient.java:816)
at org.apache.solr.client.solrj.SolrRequest.process(SolrRequest.java:194)
at org.apache.solr.client.solrj.request.UpdateRequest.commit(UpdateRequest.java:233)
at org.apache.solr.client.solrj.impl.CloudSolrClientTest.testRetryUpdatesWhenClusterStateIsStale(CloudSolrClientTest.java:844)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at org.junit.rules.ExpectedException$ExpectedExceptionStatement.evaluate(ExpectedException.java:110)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.cloud.autoscaling.HdfsAutoAddReplicasIntegrationTest.testSimple
Error Message:
Waiting for collection testSimple2 null Live Nodes: [127.0.0.1:38501_solr, 127.0.0.1:43915_solr] Last available state: DocCollection(testSimple2//collections/testSimple2/state.json/21)={ "pullReplicas":"0", "replicationFactor":"2", "shards":{ "shard1":{ "range":"80000000-ffffffff", "state":"active", "replicas":{ "core_node3":{ "dataDir":"hdfs://localhost.localdomain:33785/data/testSimple2/core_node3/data/", "base_url":"https://127.0.0.1:45301/solr", "node_name":"127.0.0.1:45301_solr", "type":"NRT", "ulogDir":"hdfs://localhost.localdomain:33785/data/testSimple2/core_node3/data/tlog", "core":"testSimple2_shard1_replica_n1", "state":"down"}, "core_node5":{ "dataDir":"hdfs://localhost.localdomain:33785/data/testSimple2/core_node5/data/", "base_url":"https://127.0.0.1:38501/solr", "node_name":"127.0.0.1:38501_solr", "type":"NRT", "ulogDir":"hdfs://localhost.localdomain:33785/data/testSimple2/core_node5/data/tlog", "core":"testSimple2_shard1_replica_n2", "state":"active", "leader":"true"}}}, "shard2":{ "range":"0-7fffffff", "state":"active", "replicas":{ "core_node7":{ "dataDir":"hdfs://localhost.localdomain:33785/data/testSimple2/core_node7/data/", "base_url":"https://127.0.0.1:45301/solr", "node_name":"127.0.0.1:45301_solr", "type":"NRT", "ulogDir":"hdfs://localhost.localdomain:33785/data/testSimple2/core_node7/data/tlog", "core":"testSimple2_shard2_replica_n4", "state":"down"}, "core_node8":{ "dataDir":"hdfs://localhost.localdomain:33785/data/testSimple2/core_node8/data/", "base_url":"https://127.0.0.1:38501/solr", "node_name":"127.0.0.1:38501_solr", "type":"NRT", "ulogDir":"hdfs://localhost.localdomain:33785/data/testSimple2/core_node8/data/tlog", "core":"testSimple2_shard2_replica_n6", "state":"active", "leader":"true"}}}}, "router":{"name":"compositeId"}, "maxShardsPerNode":"2", "autoAddReplicas":"true", "nrtReplicas":"2", "tlogReplicas":"0"}
Stack Trace:
java.lang.AssertionError: Waiting for collection testSimple2
null
Live Nodes: [127.0.0.1:38501_solr, 127.0.0.1:43915_solr]
Last available state: DocCollection(testSimple2//collections/testSimple2/state.json/21)={
"pullReplicas":"0",
"replicationFactor":"2",
"shards":{
"shard1":{
"range":"80000000-ffffffff",
"state":"active",
"replicas":{
"core_node3":{
"dataDir":"hdfs://localhost.localdomain:33785/data/testSimple2/core_node3/data/",
"base_url":"https://127.0.0.1:45301/solr",
"node_name":"127.0.0.1:45301_solr",
"type":"NRT",
"ulogDir":"hdfs://localhost.localdomain:33785/data/testSimple2/core_node3/data/tlog",
"core":"testSimple2_shard1_replica_n1",
"state":"down"},
"core_node5":{
"dataDir":"hdfs://localhost.localdomain:33785/data/testSimple2/core_node5/data/",
"base_url":"https://127.0.0.1:38501/solr",
"node_name":"127.0.0.1:38501_solr",
"type":"NRT",
"ulogDir":"hdfs://localhost.localdomain:33785/data/testSimple2/core_node5/data/tlog",
"core":"testSimple2_shard1_replica_n2",
"state":"active",
"leader":"true"}}},
"shard2":{
"range":"0-7fffffff",
"state":"active",
"replicas":{
"core_node7":{
"dataDir":"hdfs://localhost.localdomain:33785/data/testSimple2/core_node7/data/",
"base_url":"https://127.0.0.1:45301/solr",
"node_name":"127.0.0.1:45301_solr",
"type":"NRT",
"ulogDir":"hdfs://localhost.localdomain:33785/data/testSimple2/core_node7/data/tlog",
"core":"testSimple2_shard2_replica_n4",
"state":"down"},
"core_node8":{
"dataDir":"hdfs://localhost.localdomain:33785/data/testSimple2/core_node8/data/",
"base_url":"https://127.0.0.1:38501/solr",
"node_name":"127.0.0.1:38501_solr",
"type":"NRT",
"ulogDir":"hdfs://localhost.localdomain:33785/data/testSimple2/core_node8/data/tlog",
"core":"testSimple2_shard2_replica_n6",
"state":"active",
"leader":"true"}}}},
"router":{"name":"compositeId"},
"maxShardsPerNode":"2",
"autoAddReplicas":"true",
"nrtReplicas":"2",
"tlogReplicas":"0"}
at __randomizedtesting.SeedInfo.seed([1878EB209465F1E2:20CBCFDEB3962533]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.apache.solr.cloud.SolrCloudTestCase.waitForState(SolrCloudTestCase.java:269)
at org.apache.solr.cloud.autoscaling.AutoAddReplicasIntegrationTest.testSimple(AutoAddReplicasIntegrationTest.java:125)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
FAILED: org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigReplication
Error Message:
Index: 0, Size: 0
Stack Trace:
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at __randomizedtesting.SeedInfo.seed([1878EB209465F1E2:C30B075B7624CFC]:0)
at java.util.ArrayList.rangeCheck(ArrayList.java:653)
at java.util.ArrayList.get(ArrayList.java:429)
at org.apache.solr.handler.TestReplicationHandler.doTestIndexAndConfigReplication(TestReplicationHandler.java:561)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1737)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:934)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:970)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:984)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:829)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:879)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:890)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 13293 lines...]
[junit4] Suite: org.apache.solr.handler.TestReplicationHandler
[junit4] 2> Creating dataDir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/init-core-data-001
[junit4] 2> 2386258 WARN (SUITE-TestReplicationHandler-seed#[1878EB209465F1E2]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=16 numCloses=16
[junit4] 2> 2386258 INFO (SUITE-TestReplicationHandler-seed#[1878EB209465F1E2]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 2386259 INFO (SUITE-TestReplicationHandler-seed#[1878EB209465F1E2]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.SolrTestCaseJ4$SuppressSSL(bugUrl=None)
[junit4] 2> 2386259 INFO (SUITE-TestReplicationHandler-seed#[1878EB209465F1E2]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> 2386261 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.SolrTestCaseJ4 ###Starting doTestStressReplication
[junit4] 2> 2386262 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-001/collection1
[junit4] 2> 2386265 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2386266 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@557d2b{/solr,null,AVAILABLE}
[junit4] 2> 2386269 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@7dd735{HTTP/1.1,[http/1.1]}{127.0.0.1:35127}
[junit4] 2> 2386269 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.e.j.s.Server Started @2388153ms
[junit4] 2> 2386269 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=35127}
[junit4] 2> 2386270 ERROR (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2386270 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.2.0
[junit4] 2> 2386270 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 2386270 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2386270 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-10T01:02:50.970Z
[junit4] 2> 2386270 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-001/solr.xml
[junit4] 2> 2386273 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2386273 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2386273 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@17e306f, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2386300 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@17e306f
[junit4] 2> 2386308 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@17e306f
[junit4] 2> 2386308 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@17e306f
[junit4] 2> 2386310 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-001/.
[junit4] 2> 2386310 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2386310 INFO (coreLoadExecutor-4074-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2386317 INFO (coreLoadExecutor-4074-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.2.0
[junit4] 2> 2386321 INFO (coreLoadExecutor-4074-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2386323 INFO (coreLoadExecutor-4074-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 2386323 INFO (coreLoadExecutor-4074-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-001/./collection1, trusted=true
[junit4] 2> 2386323 INFO (coreLoadExecutor-4074-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@17e306f
[junit4] 2> 2386323 INFO (coreLoadExecutor-4074-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2386323 INFO (coreLoadExecutor-4074-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-001/collection1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-001/./collection1/data/]
[junit4] 2> 2386324 INFO (coreLoadExecutor-4074-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=21, maxMergeAtOnceExplicit=42, maxMergedSegmentMB=62.322265625, floorSegmentMB=0.748046875, forceMergeDeletesPctAllowed=8.87096733648037, segmentsPerTier=29.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2386350 INFO (coreLoadExecutor-4074-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2386350 INFO (coreLoadExecutor-4074-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2386350 INFO (coreLoadExecutor-4074-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1459734496, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 2386350 INFO (coreLoadExecutor-4074-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@c8ef05[collection1] main]
[junit4] 2> 2386351 INFO (coreLoadExecutor-4074-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-001/collection1/conf
[junit4] 2> 2386351 INFO (coreLoadExecutor-4074-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml,xslt/dummy.xsl
[junit4] 2> 2386351 INFO (coreLoadExecutor-4074-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2386352 INFO (searcherExecutor-4075-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@c8ef05[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2386353 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.SolrTestCaseJ4 Writing core.properties file to /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-002/collection1
[junit4] 2> 2386354 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2386355 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@2af4dd{/solr,null,AVAILABLE}
[junit4] 2> 2386355 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@e329fb{HTTP/1.1,[http/1.1]}{127.0.0.1:41947}
[junit4] 2> 2386355 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.e.j.s.Server Started @2388238ms
[junit4] 2> 2386355 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=41947}
[junit4] 2> 2386355 ERROR (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2386355 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.2.0
[junit4] 2> 2386355 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 2386355 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2386355 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-10T01:02:51.055Z
[junit4] 2> 2386356 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-002/solr.xml
[junit4] 2> 2386358 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2386358 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2386358 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@17e306f, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2386382 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@17e306f
[junit4] 2> 2386390 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@17e306f
[junit4] 2> 2386390 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@17e306f
[junit4] 2> 2386392 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-002/.
[junit4] 2> 2386392 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2386392 INFO (coreLoadExecutor-4084-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2386398 INFO (coreLoadExecutor-4084-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.2.0
[junit4] 2> 2386401 INFO (coreLoadExecutor-4084-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2386402 INFO (coreLoadExecutor-4084-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 2386402 INFO (coreLoadExecutor-4084-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-002/./collection1, trusted=true
[junit4] 2> 2386402 INFO (coreLoadExecutor-4084-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@17e306f
[junit4] 2> 2386402 INFO (coreLoadExecutor-4084-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2386402 INFO (coreLoadExecutor-4084-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-002/collection1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-002/./collection1/data/]
[junit4] 2> 2386404 INFO (coreLoadExecutor-4084-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=21, maxMergeAtOnceExplicit=42, maxMergedSegmentMB=62.322265625, floorSegmentMB=0.748046875, forceMergeDeletesPctAllowed=8.87096733648037, segmentsPerTier=29.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2386430 INFO (coreLoadExecutor-4084-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2386430 INFO (coreLoadExecutor-4084-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2386431 INFO (coreLoadExecutor-4084-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1459734496, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 2386431 INFO (coreLoadExecutor-4084-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@13b80ef[collection1] main]
[junit4] 2> 2386431 INFO (coreLoadExecutor-4084-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-002/collection1/conf
[junit4] 2> 2386432 INFO (coreLoadExecutor-4084-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 1000ms
[junit4] 2> 2386432 INFO (coreLoadExecutor-4084-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2386433 INFO (searcherExecutor-4085-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@13b80ef[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2386434 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@e329fb{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2386435 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=17280386
[junit4] 2> 2386435 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 2386435 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@b94dad: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@5ea661
[junit4] 2> 2386440 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 2386440 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1e72b3: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@ebded5
[junit4] 2> 2386444 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 2386444 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1dab3ed: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@53505f
[junit4] 2> 2386444 INFO (coreCloseExecutor-4090-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@1a2dc1e
[junit4] 2> 2386444 INFO (coreCloseExecutor-4090-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=27450398
[junit4] 2> 2386444 INFO (coreCloseExecutor-4090-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@794f65: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@1bdd826
[junit4] 2> 2386450 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@2af4dd{/solr,null,UNAVAILABLE}
[junit4] 2> 2386451 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2386452 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1514d8a{/solr,null,AVAILABLE}
[junit4] 2> 2386452 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@2b229a{HTTP/1.1,[http/1.1]}{127.0.0.1:34969}
[junit4] 2> 2386452 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.e.j.s.Server Started @2388336ms
[junit4] 2> 2386452 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-002/collection1/data, hostContext=/solr, hostPort=34969}
[junit4] 2> 2386453 ERROR (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2386453 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.2.0
[junit4] 2> 2386453 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 2386453 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2386453 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-10T01:02:51.153Z
[junit4] 2> 2386453 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-002/solr.xml
[junit4] 2> 2386455 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2386455 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2386456 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@17e306f, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2386478 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@17e306f
[junit4] 2> 2386485 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@17e306f
[junit4] 2> 2386486 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@17e306f
[junit4] 2> 2386487 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-002/.
[junit4] 2> 2386487 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2386487 INFO (coreLoadExecutor-4097-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2386504 INFO (coreLoadExecutor-4097-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.2.0
[junit4] 2> 2386517 INFO (coreLoadExecutor-4097-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2386519 INFO (coreLoadExecutor-4097-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 2386519 INFO (coreLoadExecutor-4097-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-002/./collection1, trusted=true
[junit4] 2> 2386519 INFO (coreLoadExecutor-4097-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@17e306f
[junit4] 2> 2386519 INFO (coreLoadExecutor-4097-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2386519 INFO (coreLoadExecutor-4097-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-002/collection1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-002/./collection1/data/]
[junit4] 2> 2386521 INFO (coreLoadExecutor-4097-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=21, maxMergeAtOnceExplicit=42, maxMergedSegmentMB=62.322265625, floorSegmentMB=0.748046875, forceMergeDeletesPctAllowed=8.87096733648037, segmentsPerTier=29.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2386559 INFO (coreLoadExecutor-4097-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2386559 INFO (coreLoadExecutor-4097-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2386559 INFO (coreLoadExecutor-4097-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1459734496, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 2386559 INFO (coreLoadExecutor-4097-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1cb1bca[collection1] main]
[junit4] 2> 2386560 INFO (coreLoadExecutor-4097-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-002/collection1/conf
[junit4] 2> 2386560 INFO (coreLoadExecutor-4097-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2386561 INFO (searcherExecutor-4098-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1cb1bca[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2386562 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@7dd735{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 2386562 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=31965207
[junit4] 2> 2386562 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 2386562 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@141817d: rootName = null, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@6a826
[junit4] 2> 2386566 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 2386566 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@c71283: rootName = null, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@ebded5
[junit4] 2> 2386568 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 2386568 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@17454e5: rootName = null, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@53505f
[junit4] 2> 2386569 INFO (coreCloseExecutor-4102-thread-1) [ x:collection1] o.a.s.c.SolrCore [collection1] CLOSING SolrCore org.apache.solr.core.SolrCore@16c6955
[junit4] 2> 2386569 INFO (coreCloseExecutor-4102-thread-1) [ x:collection1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.collection1, tag=23882069
[junit4] 2> 2386569 INFO (coreCloseExecutor-4102-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1a6b141: rootName = null, domain = solr.core.collection1, service url = null, agent id = null] for registry solr.core.collection1 / com.codahale.metrics.MetricRegistry@e61391
[junit4] 2> 2386573 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@557d2b{/solr,null,UNAVAILABLE}
[junit4] 2> 2386574 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2386575 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1b0e7a2{/solr,null,AVAILABLE}
[junit4] 2> 2386575 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6eadbe{HTTP/1.1,[http/1.1]}{127.0.0.1:35025}
[junit4] 2> 2386575 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.e.j.s.Server Started @2388458ms
[junit4] 2> 2386575 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {solr.data.dir=/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-001/collection1/data, hostContext=/solr, hostPort=35025}
[junit4] 2> 2386575 ERROR (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2386575 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.2.0
[junit4] 2> 2386575 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in standalone mode on port null
[junit4] 2> 2386575 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null
[junit4] 2> 2386575 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-11-10T01:02:51.275Z
[junit4] 2> 2386576 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.SolrXmlConfig Loading container configuration from /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-001/solr.xml
[junit4] 2> 2386578 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverWorkLoopDelay is ignored
[junit4] 2> 2386578 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.SolrXmlConfig Configuration parameter autoReplicaFailoverBadNodeExpiration is ignored
[junit4] 2> 2386579 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.SolrXmlConfig MBean server found: com.sun.jmx.mbeanserver.JmxMBeanServer@17e306f, but no JMX reporters were configured - adding default JMX reporter.
[junit4] 2> 2386602 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@17e306f
[junit4] 2> 2386609 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@17e306f
[junit4] 2> 2386609 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@17e306f
[junit4] 2> 2386611 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-001/.
[junit4] 2> 2386611 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.c.CorePropertiesLocator Cores are: [collection1]
[junit4] 2> 2386611 INFO (coreLoadExecutor-4109-thread-1) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 4 transient cores
[junit4] 2> 2386617 INFO (coreLoadExecutor-4109-thread-1) [ x:collection1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.2.0
[junit4] 2> 2386621 INFO (coreLoadExecutor-4109-thread-1) [ x:collection1] o.a.s.s.IndexSchema [collection1] Schema name=test
[junit4] 2> 2386622 INFO (coreLoadExecutor-4109-thread-1) [ x:collection1] o.a.s.s.IndexSchema Loaded schema test/1.2 with uniqueid field id
[junit4] 2> 2386622 INFO (coreLoadExecutor-4109-thread-1) [ x:collection1] o.a.s.c.CoreContainer Creating SolrCore 'collection1' using configuration from instancedir /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-001/./collection1, trusted=true
[junit4] 2> 2386622 INFO (coreLoadExecutor-4109-thread-1) [ x:collection1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr.core.collection1' (registry 'solr.core.collection1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@17e306f
[junit4] 2> 2386622 INFO (coreLoadExecutor-4109-thread-1) [ x:collection1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2386622 INFO (coreLoadExecutor-4109-thread-1) [ x:collection1] o.a.s.c.SolrCore [[collection1] ] Opening new SolrCore at [/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-001/collection1], dataDir=[/home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-001/./collection1/data/]
[junit4] 2> 2386624 INFO (coreLoadExecutor-4109-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=21, maxMergeAtOnceExplicit=42, maxMergedSegmentMB=62.322265625, floorSegmentMB=0.748046875, forceMergeDeletesPctAllowed=8.87096733648037, segmentsPerTier=29.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
[junit4] 2> 2386647 INFO (coreLoadExecutor-4109-thread-1) [ x:collection1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2386647 INFO (coreLoadExecutor-4109-thread-1) [ x:collection1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2386648 INFO (coreLoadExecutor-4109-thread-1) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=1459734496, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
[junit4] 2> 2386648 INFO (coreLoadExecutor-4109-thread-1) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1ede8b7[collection1] main]
[junit4] 2> 2386648 INFO (coreLoadExecutor-4109-thread-1) [ x:collection1] o.a.s.r.ManagedResourceStorage File-based storage initialized to use dir: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestReplicationHandler_1878EB209465F1E2-001/solr-instance-001/collection1/conf
[junit4] 2> 2386649 INFO (coreLoadExecutor-4109-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Replication enabled for following config files: schema.xml
[junit4] 2> 2386649 INFO (coreLoadExecutor-4109-thread-1) [ x:collection1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
[junit4] 2> 2386650 INFO (searcherExecutor-4110-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@1ede8b7[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2386651 INFO (qtp31666261-16316) [ x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4] 2> 2386651 INFO (qtp31666261-16316) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{deleteByQuery=*:*} 0 0
[junit4] 2> 2386652 INFO (qtp32220505-16296) [ x:collection1] o.a.s.u.DirectUpdateHandler2 [collection1] REMOVING ALL DOCUMENTS FROM INDEX
[junit4] 2> 2386652 INFO (qtp32220505-16296) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{deleteByQuery=*:*} 0 0
[junit4] 2> 2386652 INFO (qtp32220505-16297) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2386652 INFO (qtp32220505-16297) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@6bb159 commitCommandVersion:0
[junit4] 2> 2386653 INFO (qtp32220505-16297) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@191401f[collection1] main]
[junit4] 2> 2386654 INFO (qtp32220505-16297) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2386654 INFO (searcherExecutor-4098-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@191401f[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2386654 INFO (qtp32220505-16297) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 2
[junit4] 2> 2386656 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[0]} 0 0
[junit4] 2> 2386656 INFO (qtp31666261-16318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1]} 0 0
[junit4] 2> 2386656 INFO (qtp31666261-16311) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2]} 0 0
[junit4] 2> 2386656 INFO (qtp31666261-16316) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[3]} 0 0
[junit4] 2> 2386657 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[4]} 0 0
[junit4] 2> 2386657 INFO (qtp31666261-16318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[5]} 0 0
[junit4] 2> 2386657 INFO (qtp31666261-16311) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[6]} 0 0
[junit4] 2> 2386657 INFO (qtp31666261-16316) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[7]} 0 0
[junit4] 2> 2386657 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[8]} 0 0
[junit4] 2> 2386658 INFO (qtp31666261-16318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[9]} 0 0
[junit4] 2> 2386658 INFO (qtp31666261-16311) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[10]} 0 0
[junit4] 2> 2386658 INFO (qtp31666261-16316) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[11]} 0 0
[junit4] 2> 2386658 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[12]} 0 0
[junit4] 2> 2386659 INFO (qtp31666261-16318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[13]} 0 0
[junit4] 2> 2386659 INFO (qtp31666261-16311) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[14]} 0 0
[junit4] 2> 2386659 INFO (qtp31666261-16316) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[15]} 0 0
[junit4] 2> 2386659 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[16]} 0 0
[junit4] 2> 2386659 INFO (qtp31666261-16318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[17]} 0 0
[junit4] 2> 2386659 INFO (qtp31666261-16311) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[18]} 0 0
[junit4] 2> 2386660 INFO (qtp31666261-16316) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[19]} 0 0
[junit4] 2> 2386660 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[20]} 0 0
[junit4] 2> 2386660 INFO (qtp31666261-16318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[21]} 0 0
[junit4] 2> 2386660 INFO (qtp31666261-16311) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[22]} 0 0
[junit4] 2> 2386660 INFO (qtp31666261-16316) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[23]} 0 0
[junit4] 2> 2386660 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[24]} 0 0
[junit4] 2> 2386661 INFO (qtp31666261-16318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[25]} 0 0
[junit4] 2> 2386661 INFO (qtp31666261-16311) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[26]} 0 0
[junit4] 2> 2386661 INFO (qtp31666261-16316) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[27]} 0 0
[junit4] 2> 2386661 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[28]} 0 0
[junit4] 2> 2386661 INFO (qtp31666261-16318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[29]} 0 0
[junit4] 2> 2386661 INFO (qtp31666261-16311) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[30]} 0 0
[junit4] 2> 2386662 INFO (qtp31666261-16311) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[31]} 0 0
[junit4] 2> 2386662 INFO (qtp31666261-16316) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[32]} 0 0
[junit4] 2> 2386662 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[33]} 0 0
[junit4] 2> 2386662 INFO (qtp31666261-16318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[34]} 0 0
[junit4] 2> 2386662 INFO (qtp31666261-16311) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[35]} 0 0
[junit4] 2> 2386663 INFO (qtp31666261-16316) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[36]} 0 0
[junit4] 2> 2386663 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[37]} 0 0
[junit4] 2> 2386663 INFO (qtp31666261-16318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[38]} 0 0
[junit4] 2> 2386663 INFO (qtp31666261-16311) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[39]} 0 0
[junit4] 2> 2386663 INFO (qtp31666261-16316) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[40]} 0 0
[junit4] 2> 2386664 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[41]} 0 0
[junit4] 2> 2386664 INFO (qtp31666261-16318) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[42]} 0 0
[junit4] 2> 2386664 INFO (qtp31666261-16311) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[43]} 0 0
[junit4] 2> 2386664 INFO (qtp31666261-16316) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[44]} 0 0
[junit4] 2> 2386664 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2386664 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@3baf7f commitCommandVersion:0
[junit4] 2> 2386667 INFO (qtp31666261-16317) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@bcd9ae[collection1] main]
[junit4] 2> 2386667 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2386668 INFO (searcherExecutor-4110-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@bcd9ae[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.2.0):C45)))}
[junit4] 2> 2386668 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 3
[junit4] 2> 2386669 INFO (qtp31666261-16318) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/select params={q=*:*&sort=id+desc&wt=javabin&version=2} hits=45 status=0 QTime=0
[junit4] 2> 2386669 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.h.TestReplicationHandler Waited for 0ms and found 45 docs
[junit4] 2> 2386671 INFO (qtp32220505-16291) [ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={core=collection1&action=status&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2386675 WARN (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher 'masterUrl' must be specified without the /replication suffix
[junit4] 2> 2386676 INFO (qtp31666261-16312) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2386676 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Master's generation: 2
[junit4] 2> 2386676 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Master's version: 1510275771364
[junit4] 2> 2386676 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Slave's generation: 2
[junit4] 2> 2386676 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Slave's version: 1510275771352
[junit4] 2> 2386676 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Starting replication process
[junit4] 2> 2386678 INFO (qtp31666261-16316) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&tlogFiles=false&wt=javabin&version=2&command=filelist} status=0 QTime=1
[junit4] 2> 2386678 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Number of files in latest index in master: 10
[junit4] 2> 2386678 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Starting download (fullCopy=true) to MockDirectoryWrapper(RAMDirectory@ef611c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@42e694)
[junit4] 2> 2386679 INFO (qtp31666261-16317) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2386679 INFO (qtp31666261-16318) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2386680 INFO (qtp31666261-16312) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2386681 INFO (qtp31666261-16316) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2386681 INFO (qtp31666261-16317) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2386682 INFO (qtp31666261-16318) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2386682 INFO (qtp31666261-16312) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2386683 INFO (qtp31666261-16316) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fnm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2386683 INFO (qtp31666261-16317) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2386683 WARN (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher File segments_2 did not match. expected checksum is 3811246498 and actual is checksum 3292434679. expected length is 185 and actual length is 117
[junit4] 2> 2386684 INFO (qtp31666261-16318) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={generation=2&qt=/replication&file=segments_2&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2386684 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher Total time taken for download (fullCopy=true,bytesDownloaded=2671) : 0 secs (null bytes/sec) to MockDirectoryWrapper(RAMDirectory@ef611c lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@42e694)
[junit4] 2> 2386684 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.c.SolrCore Updating index properties... index=index.20171110090251378
[junit4] 2> 2386685 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.h.IndexFetcher removing old index directory MockDirectoryWrapper(NRTCachingDirectory(RAMDirectory@9f9653 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@8187c3; maxCacheMB=0.77734375 maxMergeSizeMB=0.203125))
[junit4] 2> 2386685 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.RandomMergePolicy RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=12, maxMergeAtOnceExplicit=36, maxMergedSegmentMB=76.4912109375, floorSegmentMB=0.6064453125, forceMergeDeletesPctAllowed=23.347788740695595, segmentsPerTier=26.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.42262006270502384
[junit4] 2> 2386685 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 2386685 INFO (explicit-fetchindex-cmd) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@22d2e2[collection1] main]
[junit4] 2> 2386686 INFO (searcherExecutor-4098-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@22d2e2[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.2.0):C45)))}
[junit4] 2> 2386686 INFO (qtp32220505-16291) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={wait=true&masterUrl=http://127.0.0.1:35025/solr/collection1/replication&command=fetchindex} status=0 QTime=11
[junit4] 2> 2386688 INFO (qtp32220505-16298) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/select params={q=*:*&sort=id+desc&wt=javabin&version=2} hits=45 status=0 QTime=0
[junit4] 2> 2386688 INFO (TEST-TestReplicationHandler.doTestStressReplication-seed#[1878EB209465F1E2]) [ ] o.a.s.h.TestReplicationHandler Waited for 0ms and found 45 docs
[junit4] 2> 2386688 INFO (qtp31666261-16311) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=getDetails&qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2386689 INFO (qtp31666261-16316) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=getDetails&qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2386689 INFO (qtp31666261-16312) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={slave=false&qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2386690 INFO (qtp32220505-16292) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=getDetails&qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2386690 INFO (qtp31666261-16317) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=assertVersions&qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2386691 INFO (qtp32220505-16293) [ x:collection1] o.a.s.c.S.Request [collection1] webapp=/solr path=/replication params={_trace=assertVersions&qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2386691 INFO (qtp32220505-16293) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[45]} 0 0
[junit4] 2> 2386692 INFO (qtp32220505-16293) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[46]} 0 0
[junit4] 2> 2386692 INFO (qtp32220505-16292) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[47]} 0 0
[junit4] 2> 2386692 INFO (qtp32220505-16291) [ x:collection1] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2386692 INFO (qtp32220505-16291) [ x:collection1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@817a32 commitCommandVersion:0
[junit4] 2> 2386693 INFO (qtp32220505-16291) [ x:collection1] o.a.s.s.SolrIndexSearcher Opening [Searcher@e02803[collection1] main]
[junit4] 2> 2386693 INFO (qtp32220505-16291) [ x:collection1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2386694 INFO (searcherExecutor-4098-thread-1-processing-x:collection1) [ x:collection1] o.a.s.c.SolrCore [collection1] Registered new searcher Searcher@e02803[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.2.0):C45) Uninverting(_1(7.2.0):C3)))}
[junit4] 2> 2386694 INFO (qtp32220505-16291) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 1
[junit4] 2> 2386695 INFO (qtp31666261-16311) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[48]} 0 0
[junit4] 2> 2386696 INFO (qtp31666261-16316) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[49]} 0 0
[junit4] 2> 2386696 INFO (qtp31666261-16312) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[50]} 0 0
[junit4] 2> 2386696 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[51]} 0 0
[junit4] 2> 2386696 INFO (qtp31666261-16311) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[52]} 0 0
[junit4] 2> 2386696 INFO (qtp31666261-16316) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[53]} 0 0
[junit4] 2> 2386697 INFO (qtp31666261-16312) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[54]} 0 0
[junit4] 2> 2386697 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[55]} 0 0
[junit4] 2> 2386697 INFO (qtp31666261-16311) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[56]} 0 0
[junit4] 2> 2386697 INFO (qtp31666261-16316) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[57]} 0 0
[junit4] 2> 2386697 INFO (qtp31666261-16312) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[58]} 0 0
[junit4] 2> 2386697 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[59]} 0 0
[junit4] 2> 2386698 INFO (qtp31666261-16311) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[60]} 0 0
[junit4] 2> 2386698 INFO (qtp31666261-16316) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[61]} 0 0
[junit4] 2> 2386698 INFO (qtp31666261-16312) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[62]} 0 0
[junit4] 2> 2386698 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[63]} 0 0
[junit4] 2> 2386698 INFO (qtp31666261-16311) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[64]} 0 0
[junit4] 2> 2386698 INFO (qtp31666261-16316) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[65]} 0 0
[junit4] 2> 2386699 INFO (qtp31666261-16312) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[66]} 0 0
[junit4] 2> 2386699 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[67]} 0 0
[junit4] 2> 2386699 INFO (qtp31666261-16311) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[68]} 0 0
[junit4] 2> 2386699 INFO (qtp31666261-16316) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[69]} 0 0
[junit4] 2> 2386699 INFO (qtp31666261-16312) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[70]} 0 0
[junit4] 2> 2386699 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[71]} 0 0
[junit4] 2> 2386700 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[72]} 0 0
[junit4] 2> 2386700 INFO (qtp31666261-16316) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[73]} 0 0
[junit4] 2> 2386700 INFO (qtp31666261-16312) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[74]} 0 0
[junit4] 2> 2386700 INFO (qtp31666261-16311) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[75]} 0 0
[junit4] 2> 2386700 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[76]} 0 0
[junit4] 2> 2386700 INFO (qtp31666261-16316) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[77]} 0 0
[junit4] 2> 2386701 INFO (qtp31666261-16312) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[78]} 0 0
[junit4] 2> 2386701 INFO (qtp31666261-16311) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[79]} 0 0
[junit4] 2> 2386701 INFO (qtp31666261-16317) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[80]} 0 0
[junit4] 2> 2386701 INFO (qtp31666261-16316) [ x:collection1] o.a.s.u.p.LogUpdateProcessorFactory [collection1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[81]} 0 0
[junit4] 2> 2386701 INFO (qtp31666261-16312) [ x:collection1] o.a.s.u.p.LogUpdate
[...truncated too long message...]
FO (coreCloseExecutor-148-thread-7) [n:127.0.0.1:38643_solr c:multicollection2 s:shard1 r:core_node3 x:multicollection2_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1e7b0e1: rootName = solr_38643, domain = solr.core.multicollection2.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.multicollection2.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@1576245
[junit4] 2> 41975 INFO (coreCloseExecutor-149-thread-7) [n:127.0.0.1:39311_solr c:multicollection2 s:shard2 r:core_node4 x:multicollection2_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.multicollection2.shard2.leader, tag=10572753
[junit4] 2> 41976 INFO (coreCloseExecutor-150-thread-1) [n:127.0.0.1:45875_solr c:collection1 s:shard1 r:core_node3 x:collection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.collection1.shard1.leader, tag=3231757
[junit4] 2> 41977 INFO (coreCloseExecutor-150-thread-8) [n:127.0.0.1:45875_solr c:multicollection1 s:shard2 r:core_node4 x:multicollection1_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.multicollection1.shard2.replica_n2, tag=5652558
[junit4] 2> 41977 INFO (coreCloseExecutor-150-thread-8) [n:127.0.0.1:45875_solr c:multicollection1 s:shard2 r:core_node4 x:multicollection1_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1478ddc: rootName = solr_45875, domain = solr.core.multicollection1.shard2.replica_n2, service url = null, agent id = null] for registry solr.core.multicollection1.shard2.replica_n2 / com.codahale.metrics.MetricRegistry@4005a1
[junit4] 2> 41985 INFO (coreCloseExecutor-150-thread-7) [n:127.0.0.1:45875_solr c:foo s:shard1 r:core_node5 x:foo_shard1_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.foo.shard1.leader, tag=4344577
[junit4] 2> 41985 INFO (coreCloseExecutor-150-thread-2) [n:127.0.0.1:45875_solr c:localShardsTestColl s:shard2 r:core_node13 x:localShardsTestColl_shard2_replica_n9] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard2.leader, tag=21635509
[junit4] 2> 41985 INFO (coreCloseExecutor-150-thread-3) [n:127.0.0.1:45875_solr c:localShardsTestColl s:shard3 r:core_node18 x:localShardsTestColl_shard3_replica_n16] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard3.leader, tag=3295946
[junit4] 2> 41985 INFO (coreCloseExecutor-150-thread-6) [n:127.0.0.1:45875_solr c:2nd_collection s:shard2 r:core_node4 x:2nd_collection_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.2nd_collection.shard2.leader, tag=314453
[junit4] 2> 41986 INFO (coreCloseExecutor-150-thread-8) [n:127.0.0.1:45875_solr c:multicollection1 s:shard2 r:core_node4 x:multicollection1_shard2_replica_n2] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.multicollection1.shard2.leader, tag=5652558
[junit4] 2> 41987 INFO (coreCloseExecutor-148-thread-7) [n:127.0.0.1:38643_solr c:multicollection2 s:shard1 r:core_node3 x:multicollection2_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.multicollection2.shard1.leader, tag=26877364
[junit4] 2> 41987 INFO (coreCloseExecutor-148-thread-1) [n:127.0.0.1:38643_solr c:localShardsTestColl s:shard1 r:core_node3 x:localShardsTestColl_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard1.leader, tag=9995012
[junit4] 2> 41987 INFO (coreCloseExecutor-148-thread-3) [n:127.0.0.1:38643_solr c:localShardsTestColl s:shard3 r:core_node15 x:localShardsTestColl_shard3_replica_n11] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.localShardsTestColl.shard3.leader, tag=13617532
[junit4] 2> 41987 INFO (coreCloseExecutor-148-thread-4) [n:127.0.0.1:38643_solr c:2nd_collection s:shard1 r:core_node3 x:2nd_collection_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.2nd_collection.shard1.leader, tag=27512977
[junit4] 2> 41987 INFO (coreCloseExecutor-148-thread-5) [n:127.0.0.1:38643_solr c:foo s:shard1 r:core_node3 x:foo_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.foo.shard1.leader, tag=20304667
[junit4] 2> 41987 INFO (coreCloseExecutor-148-thread-6) [n:127.0.0.1:38643_solr c:multicollection1 s:shard1 r:core_node3 x:multicollection1_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.multicollection1.shard1.leader, tag=9822811
[junit4] 2> 42014 INFO (jetty-closer-2-thread-3) [ ] o.a.s.c.Overseer Overseer (id=98977504330711048-127.0.0.1:45875_solr-n_0000000000) closing
[junit4] 2> 42014 INFO (OverseerStateUpdate-98977504330711048-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:45875_solr
[junit4] 2> 42014 WARN (OverseerAutoScalingTriggerThread-98977504330711048-127.0.0.1:45875_solr-n_0000000000) [n:127.0.0.1:45875_solr ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
[junit4] 2> 42018 INFO (jetty-closer-2-thread-2) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@9eb5f3{/solr,null,UNAVAILABLE}
[junit4] 2> 42019 INFO (jetty-closer-2-thread-3) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@175f376{/solr,null,UNAVAILABLE}
[junit4] 2> 42019 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15fa3840a630008, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 42020 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15fa3840a630009, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 43521 INFO (jetty-closer-2-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@984122{/solr,null,UNAVAILABLE}
[junit4] 2> 43521 ERROR (SUITE-CloudSolrClientTest-seed#[1D670CA03ECF9456]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 43522 INFO (SUITE-CloudSolrClientTest-seed#[1D670CA03ECF9456]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40023 40023
[junit4] 2> 48556 INFO (Thread-1) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:40023 40023
[junit4] 2> 48557 WARN (Thread-1) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/aliases.json
[junit4] 2> 4 /solr/configs/conf
[junit4] 2> 3 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 21 /solr/collections/foo/state.json
[junit4] 2> 21 /solr/collections/localShardsTestColl/state.json
[junit4] 2> 16 /solr/collections/stale_state_test_col/state.json
[junit4] 2> 14 /solr/collections/collection1/state.json
[junit4] 2> 12 /solr/collections/2nd_collection/state.json
[junit4] 2> 12 /solr/collections/multicollection1/state.json
[junit4] 2> 12 /solr/collections/multicollection2/state.json
[junit4] 2> 10 /solr/collections/nemesis/state.json
[junit4] 2> 8 /solr/clusterprops.json
[junit4] 2> 8 /solr/clusterstate.json
[junit4] 2> 6 /solr/collections/overwrite/state.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 8 /solr/live_nodes
[junit4] 2> 8 /solr/collections
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: /home/jenkins/workspace/Lucene-Solr-7.x-Linux/solr/build/solr-solrj/test/J2/temp/solr.client.solrj.impl.CloudSolrClientTest_1D670CA03ECF9456-001
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {multiDefault=Lucene50(blocksize=128), a_t=FST50, title_s=Lucene50(blocksize=128), id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128))), text=PostingsFormat(name=Memory)}, docValues:{_version_=DocValuesFormat(name=Lucene70), multiDefault=DocValuesFormat(name=Asserting), title_s=DocValuesFormat(name=Asserting), intDefault=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Lucene70), timestamp=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=112, maxMBSortInHeap=5.853434003793147, sim=RandomSimilarity(queryNorm=false): {}, locale=he, timezone=Pacific/Kwajalein
[junit4] 2> NOTE: Linux 4.10.0-37-generic i386/Oracle Corporation 1.8.0_144 (32-bit)/cpus=8,threads=1,free=146382192,total=295698432
[junit4] 2> NOTE: All tests run in this JVM: [CloudSolrClientTest]
[junit4] Completed [11/146 (1!)] on J2 in 49.41s, 16 tests, 1 error <<< FAILURES!
[...truncated 42603 lines...]