You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2017/08/31 07:26:32 UTC
[JENKINS-EA] Lucene-Solr-master-Windows (32bit/jdk-9-ea+181) -
Build # 6859 - Still Failing!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6859/
Java: 32bit/jdk-9-ea+181 -server -XX:+UseParallelGC --illegal-access=deny
1 tests failed.
FAILED: org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster
Error Message:
Document mismatch on target after sync expected:<2000> but was:<1100>
Stack Trace:
java.lang.AssertionError: Document mismatch on target after sync expected:<2000> but was:<1100>
at __randomizedtesting.SeedInfo.seed([B4D033E20BD53376:609578BBEC83808D]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster(CdcrBootstrapTest.java:309)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.base/java.lang.Thread.run(Thread.java:844)
Build Log:
[...truncated 11607 lines...]
[junit4] Suite: org.apache.solr.cloud.CdcrBootstrapTest
[junit4] 2> 504614 INFO (SUITE-CdcrBootstrapTest-seed#[B4D033E20BD53376]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrBootstrapTest_B4D033E20BD53376-001\init-core-data-001
[junit4] 2> 504615 WARN (SUITE-CdcrBootstrapTest-seed#[B4D033E20BD53376]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 504615 INFO (SUITE-CdcrBootstrapTest-seed#[B4D033E20BD53376]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 504616 INFO (SUITE-CdcrBootstrapTest-seed#[B4D033E20BD53376]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0)
[junit4] 2> 504617 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[B4D033E20BD53376]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testBootstrapWithSourceCluster
[junit4] 2> 504619 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[B4D033E20BD53376]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrBootstrapTest_B4D033E20BD53376-001\cdcr-target-001
[junit4] 2> 504619 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[B4D033E20BD53376]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 504620 INFO (Thread-1420) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 504620 INFO (Thread-1420) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 504624 ERROR (Thread-1420) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 504720 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[B4D033E20BD53376]) [ ] o.a.s.c.ZkTestServer start zk server on port:51714
[junit4] 2> 504729 INFO (jetty-launcher-2224-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 504730 INFO (jetty-launcher-2224-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@7a821c{/solr,null,AVAILABLE}
[junit4] 2> 504734 INFO (jetty-launcher-2224-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@721427{HTTP/1.1,[http/1.1]}{127.0.0.1:?????}
[junit4] 2> 504734 INFO (jetty-launcher-2224-thread-1) [ ] o.e.j.s.Server Started @??????ms
[junit4] 2> 504734 INFO (jetty-launcher-2224-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51718}
[junit4] 2> 504734 ERROR (jetty-launcher-2224-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 504734 INFO (jetty-launcher-2224-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 504734 INFO (jetty-launcher-2224-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 504734 INFO (jetty-launcher-2224-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 504734 INFO (jetty-launcher-2224-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-31T06:33:38.476788900Z
[junit4] 2> 504738 INFO (jetty-launcher-2224-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 504746 INFO (jetty-launcher-2224-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51714/solr
[junit4] 2> 504880 INFO (jetty-launcher-2224-thread-1) [ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 504880 INFO (jetty-launcher-2224-thread-1) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51718_solr
[junit4] 2> 504882 INFO (jetty-launcher-2224-thread-1) [ ] o.a.s.c.Overseer Overseer (id=98576709606768643-127.0.0.1:51718_solr-n_0000000000) starting
[junit4] 2> 504890 INFO (jetty-launcher-2224-thread-1) [ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51718_solr
[junit4] 2> 504892 INFO (zkCallback-2231-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 505016 INFO (jetty-launcher-2224-thread-1) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51718.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@135374d
[junit4] 2> 505025 INFO (jetty-launcher-2224-thread-1) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51718.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@135374d
[junit4] 2> 505025 INFO (jetty-launcher-2224-thread-1) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51718.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@135374d
[junit4] 2> 505026 INFO (jetty-launcher-2224-thread-1) [ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrBootstrapTest_B4D033E20BD53376-001\cdcr-target-001\node1\.
[junit4] 1> Target zkHost = 127.0.0.1:51714/solr
[junit4] 2> 505050 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[B4D033E20BD53376]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrBootstrapTest_B4D033E20BD53376-001\cdcr-source-001
[junit4] 2> 505050 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[B4D033E20BD53376]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 505050 INFO (Thread-1430) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 505050 INFO (Thread-1430) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 505055 ERROR (Thread-1430) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 505151 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[B4D033E20BD53376]) [ ] o.a.s.c.ZkTestServer start zk server on port:51738
[junit4] 2> 505159 INFO (jetty-launcher-2236-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 505160 INFO (jetty-launcher-2236-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@15623d7{/solr,null,AVAILABLE}
[junit4] 2> 505161 INFO (jetty-launcher-2236-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@11df35a{HTTP/1.1,[http/1.1]}{127.0.0.1:?????}
[junit4] 2> 505161 INFO (jetty-launcher-2236-thread-1) [ ] o.e.j.s.Server Started @??????ms
[junit4] 2> 505161 INFO (jetty-launcher-2236-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51742}
[junit4] 2> 505161 ERROR (jetty-launcher-2236-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 505162 INFO (jetty-launcher-2236-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 8.0.0
[junit4] 2> 505162 INFO (jetty-launcher-2236-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 505162 INFO (jetty-launcher-2236-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 505162 INFO (jetty-launcher-2236-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-31T06:33:38.904219200Z
[junit4] 2> 505166 INFO (jetty-launcher-2236-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 505173 INFO (jetty-launcher-2236-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51738/solr
[junit4] 2> 505308 INFO (jetty-launcher-2236-thread-1) [ ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 505309 INFO (jetty-launcher-2236-thread-1) [ ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51742_solr
[junit4] 2> 505311 INFO (jetty-launcher-2236-thread-1) [ ] o.a.s.c.Overseer Overseer (id=98576709635014659-127.0.0.1:51742_solr-n_0000000000) starting
[junit4] 2> 505319 INFO (jetty-launcher-2236-thread-1) [ ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51742_solr
[junit4] 2> 505322 INFO (OverseerStateUpdate-98576709635014659-127.0.0.1:51742_solr-n_0000000000) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 505517 INFO (jetty-launcher-2236-thread-1) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51742.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@135374d
[junit4] 2> 505525 INFO (jetty-launcher-2236-thread-1) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51742.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@135374d
[junit4] 2> 505525 INFO (jetty-launcher-2236-thread-1) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51742.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@135374d
[junit4] 2> 505526 INFO (jetty-launcher-2236-thread-1) [ ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrBootstrapTest_B4D033E20BD53376-001\cdcr-source-001\node1\.
[junit4] 2> 505569 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[B4D033E20BD53376]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 505569 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[B4D033E20BD53376]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:51738/solr ready
[junit4] 2> 505574 INFO (qtp5308053-7182) [ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=cdcr-source&name=cdcr-source&nrtReplicas=1&action=CREATE&numShards=1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 505577 INFO (OverseerThreadFactory-2546-thread-1) [ ] o.a.s.c.CreateCollectionCmd Create collection cdcr-source
[junit4] 2> 505686 INFO (OverseerStateUpdate-98576709635014659-127.0.0.1:51742_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"cdcr-source",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"cdcr-source_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:51742/solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 505888 INFO (qtp5308053-7184) [ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=cdcr-source&newCollection=true&collection=cdcr-source&version=2&replicaType=NRT&coreNodeName=core_node2&name=cdcr-source_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
[junit4] 2> 505889 INFO (qtp5308053-7184) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 505996 INFO (zkCallback-2243-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 505996 INFO (zkCallback-2243-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 506903 INFO (qtp5308053-7184) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 506908 INFO (qtp5308053-7184) [ ] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema.xml instead
[junit4] 2> 506912 INFO (qtp5308053-7184) [ ] o.a.s.s.IndexSchema [cdcr-source_shard1_replica_n1] Schema name=minimal
[junit4] 2> 506914 INFO (qtp5308053-7184) [ ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 506914 INFO (qtp5308053-7184) [ ] o.a.s.s.ManagedIndexSchema Created and persisted managed schema znode at /configs/cdcr-source/managed-schema
[junit4] 2> 506918 INFO (qtp5308053-7184) [ ] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, renamed the non-managed schema /configs/cdcr-source/schema.xml to /configs/cdcr-source/schema.xml.bak
[junit4] 2> 506918 INFO (qtp5308053-7184) [ ] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-source_shard1_replica_n1' using configuration from collection cdcr-source, trusted=true
[junit4] 2> 506918 INFO (qtp5308053-7184) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51742.solr.core.cdcr-source.shard1.replica_n1' (registry 'solr.core.cdcr-source.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@135374d
[junit4] 2> 506918 INFO (qtp5308053-7184) [ ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 506919 INFO (qtp5308053-7184) [ ] o.a.s.c.SolrCore [[cdcr-source_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrBootstrapTest_B4D033E20BD53376-001\cdcr-source-001\node1\cdcr-source_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrBootstrapTest_B4D033E20BD53376-001\cdcr-source-001\node1\.\cdcr-source_shard1_replica_n1\data\]
[junit4] 2> 507034 INFO (qtp5308053-7184) [ ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 507034 INFO (qtp5308053-7184) [ ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 507036 INFO (qtp5308053-7184) [ ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 507036 INFO (qtp5308053-7184) [ ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 507039 INFO (qtp5308053-7184) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1edf611[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 507041 INFO (qtp5308053-7184) [ ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-source
[junit4] 2> 507042 INFO (qtp5308053-7184) [ ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-source
[junit4] 2> 507042 INFO (qtp5308053-7184) [ ] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-source/managed-schema
[junit4] 2> 507042 INFO (qtp5308053-7184) [ ] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 507043 INFO (qtp5308053-7184) [ ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 507050 INFO (qtp5308053-7184) [ ] o.a.s.h.CdcrBufferStateManager Created znode /collections/cdcr-source/cdcr/state/buffer
[junit4] 2> 507054 INFO (qtp5308053-7184) [ ] o.a.s.h.CdcrProcessStateManager Created znode /collections/cdcr-source/cdcr/state/process
[junit4] 2> 507068 INFO (searcherExecutor-2549-thread-1) [ ] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@1edf611[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 507069 INFO (qtp5308053-7184) [ ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577227356273115136
[junit4] 2> 507076 INFO (qtp5308053-7184) [ ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 507076 INFO (qtp5308053-7184) [ ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 507076 INFO (qtp5308053-7184) [ ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51742/solr/cdcr-source_shard1_replica_n1/
[junit4] 2> 507076 INFO (qtp5308053-7184) [ ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 507076 INFO (qtp5308053-7184) [ ] o.a.s.c.SyncStrategy http://127.0.0.1:51742/solr/cdcr-source_shard1_replica_n1/ has no replicas
[junit4] 2> 507076 INFO (qtp5308053-7184) [ ] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 507080 INFO (zkCallback-2243-thread-2) [ ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-source:shard1
[junit4] 2> 507082 INFO (qtp5308053-7184) [ ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51742/solr/cdcr-source_shard1_replica_n1/ shard1
[junit4] 2> 507185 INFO (zkCallback-2243-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 507185 INFO (zkCallback-2243-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 507233 INFO (qtp5308053-7184) [ ] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 507235 INFO (qtp5308053-7184) [ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=cdcr-source&newCollection=true&collection=cdcr-source&version=2&replicaType=NRT&coreNodeName=core_node2&name=cdcr-source_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=1346
[junit4] 2> 507238 INFO (qtp5308053-7182) [ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 507338 INFO (zkCallback-2243-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 507338 INFO (zkCallback-2243-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 507579 INFO (OverseerCollectionConfigSetProcessor-98576709635014659-127.0.0.1:51742_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 508239 INFO (qtp5308053-7182) [ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=cdcr-source&name=cdcr-source&nrtReplicas=1&action=CREATE&numShards=1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2} status=0 QTime=2664
[junit4] 1> Adding 10 docs with commit=true, numDocs=100
[junit4] 2> 508257 INFO (qtp5308053-7185) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=14
[junit4] 2> 508259 INFO (qtp5308053-7186) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577227357520920576,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 508259 INFO (qtp5308053-7186) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@e41152 commitCommandVersion:1577227357520920576
[junit4] 2> 508280 INFO (qtp5308053-7186) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@928b7a[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 508281 INFO (qtp5308053-7186) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 508283 INFO (searcherExecutor-2549-thread-1) [ ] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@928b7a[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C100)))}
[junit4] 2> 508283 INFO (qtp5308053-7186) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=24
[junit4] 1> Adding 10 docs with commit=true, numDocs=200
[junit4] 2> 508293 INFO (qtp5308053-7187) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=8
[junit4] 2> 508294 INFO (qtp5308053-7180) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577227357557620736,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 508294 INFO (qtp5308053-7180) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@e41152 commitCommandVersion:1577227357557620736
[junit4] 2> 508317 INFO (qtp5308053-7180) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@5c7b78[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 508318 INFO (qtp5308053-7180) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 508319 INFO (searcherExecutor-2549-thread-1) [ ] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@5c7b78[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C100) Uninverting(_1(8.0.0):C100)))}
[junit4] 2> 508320 INFO (qtp5308053-7180) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=26
[junit4] 1> Adding 10 docs with commit=true, numDocs=300
[junit4] 2> 508329 INFO (qtp5308053-7183) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=7
[junit4] 2> 508330 INFO (qtp5308053-7184) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577227357595369472,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 508330 INFO (qtp5308053-7184) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@e41152 commitCommandVersion:1577227357595369472
[junit4] 2> 508361 INFO (qtp5308053-7184) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@10d1a99[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 508362 INFO (qtp5308053-7184) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 508363 INFO (searcherExecutor-2549-thread-1) [ ] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@10d1a99[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C100) Uninverting(_1(8.0.0):C100) Uninverting(_2(8.0.0):C100)))}
[junit4] 2> 508363 INFO (qtp5308053-7184) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=33
[junit4] 1> Adding 10 docs with commit=true, numDocs=400
[junit4] 2> 508371 INFO (qtp5308053-7182) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=6
[junit4] 2> 508372 INFO (qtp5308053-7185) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577227357639409664,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 508372 INFO (qtp5308053-7185) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@e41152 commitCommandVersion:1577227357639409664
[junit4] 2> 508403 INFO (qtp5308053-7185) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@bf3ad3[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 508404 INFO (qtp5308053-7185) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 508406 INFO (searcherExecutor-2549-thread-1) [ ] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@bf3ad3[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C100) Uninverting(_1(8.0.0):C100) Uninverting(_2(8.0.0):C100) Uninverting(_3(8.0.0):C100)))}
[junit4] 2> 508406 INFO (qtp5308053-7185) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=33
[junit4] 1> Adding 10 docs with commit=true, numDocs=500
[junit4] 2> 508416 INFO (qtp5308053-7186) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=9
[junit4] 2> 508417 INFO (qtp5308053-7187) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577227357686595584,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 508417 INFO (qtp5308053-7187) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@e41152 commitCommandVersion:1577227357686595584
[junit4] 2> 508461 INFO (qtp5308053-7187) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@adffe3[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 508462 INFO (qtp5308053-7187) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 508463 INFO (searcherExecutor-2549-thread-1) [ ] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@adffe3[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C100) Uninverting(_1(8.0.0):C100) Uninverting(_2(8.0.0):C100) Uninverting(_3(8.0.0):C100) Uninverting(_4(8.0.0):C100)))}
[junit4] 2> 508463 INFO (qtp5308053-7187) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=45
[junit4] 1> Adding 10 docs with commit=true, numDocs=600
[junit4] 2> 508470 INFO (qtp5308053-7180) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=6
[junit4] 2> 508471 INFO (qtp5308053-7183) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577227357743218688,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 508471 INFO (qtp5308053-7183) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@e41152 commitCommandVersion:1577227357743218688
[junit4] 2> 508503 INFO (qtp5308053-7183) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1b1c331[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 508504 INFO (qtp5308053-7183) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 508505 INFO (searcherExecutor-2549-thread-1) [ ] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@1b1c331[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C100) Uninverting(_1(8.0.0):C100) Uninverting(_2(8.0.0):C100) Uninverting(_3(8.0.0):C100) Uninverting(_4(8.0.0):C100) Uninverting(_5(8.0.0):C100)))}
[junit4] 2> 508505 INFO (qtp5308053-7183) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=34
[junit4] 1> Adding 10 docs with commit=true, numDocs=700
[junit4] 2> 508514 INFO (qtp5308053-7184) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=7
[junit4] 2> 508514 INFO (qtp5308053-7182) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577227357788307456,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 508514 INFO (qtp5308053-7182) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@e41152 commitCommandVersion:1577227357788307456
[junit4] 2> 508558 INFO (qtp5308053-7182) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@198e78e[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 508558 INFO (qtp5308053-7182) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 508560 INFO (searcherExecutor-2549-thread-1) [ ] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@198e78e[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C100) Uninverting(_1(8.0.0):C100) Uninverting(_2(8.0.0):C100) Uninverting(_3(8.0.0):C100) Uninverting(_4(8.0.0):C100) Uninverting(_5(8.0.0):C100) Uninverting(_6(8.0.0):C100)))}
[junit4] 2> 508560 INFO (qtp5308053-7182) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=45
[junit4] 1> Adding 10 docs with commit=true, numDocs=800
[junit4] 2> 508567 INFO (qtp5308053-7185) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=6
[junit4] 2> 508568 INFO (qtp5308053-7186) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577227357844930560,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 508568 INFO (qtp5308053-7186) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@e41152 commitCommandVersion:1577227357844930560
[junit4] 2> 508611 INFO (qtp5308053-7186) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1782cf9[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 508611 INFO (qtp5308053-7186) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 508613 INFO (searcherExecutor-2549-thread-1) [ ] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@1782cf9[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C100) Uninverting(_1(8.0.0):C100) Uninverting(_2(8.0.0):C100) Uninverting(_3(8.0.0):C100) Uninverting(_4(8.0.0):C100) Uninverting(_5(8.0.0):C100) Uninverting(_6(8.0.0):C100) Uninverting(_7(8.0.0):C100)))}
[junit4] 2> 508613 INFO (qtp5308053-7186) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=45
[junit4] 1> Adding 10 docs with commit=true, numDocs=900
[junit4] 2> 508621 INFO (qtp5308053-7187) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=6
[junit4] 2> 508622 INFO (qtp5308053-7180) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577227357901553664,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 508622 INFO (qtp5308053-7180) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@e41152 commitCommandVersion:1577227357901553664
[junit4] 2> 508677 INFO (qtp5308053-7180) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@a9a1e4[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 508678 INFO (qtp5308053-7180) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 508680 INFO (searcherExecutor-2549-thread-1) [ ] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@a9a1e4[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C100) Uninverting(_1(8.0.0):C100) Uninverting(_2(8.0.0):C100) Uninverting(_3(8.0.0):C100) Uninverting(_4(8.0.0):C100) Uninverting(_5(8.0.0):C100) Uninverting(_6(8.0.0):C100) Uninverting(_7(8.0.0):C100) Uninverting(_8(8.0.0):C100)))}
[junit4] 2> 508680 INFO (qtp5308053-7180) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=58
[junit4] 1> Adding 10 docs with commit=true, numDocs=1000
[junit4] 2> 508687 INFO (qtp5308053-7183) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=5
[junit4] 2> 508688 INFO (qtp5308053-7184) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577227357969711104,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 508688 INFO (qtp5308053-7184) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@e41152 commitCommandVersion:1577227357969711104
[junit4] 2> 508736 INFO (qtp5308053-7184) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@13e8aed[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 508737 INFO (qtp5308053-7184) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 508740 INFO (searcherExecutor-2549-thread-1) [ ] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@13e8aed[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(8.0.0):C100) Uninverting(_1(8.0.0):C100) Uninverting(_2(8.0.0):C100) Uninverting(_3(8.0.0):C100) Uninverting(_4(8.0.0):C100) Uninverting(_5(8.0.0):C100) Uninverting(_6(8.0.0):C100) Uninverting(_7(8.0.0):C100) Uninverting(_8(8.0.0):C100) Uninverting(_9(8.0.0):C100)))}
[junit4] 2> 508742 INFO (qtp5308053-7184) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=54
[junit4] 2> 508745 INFO (qtp5308053-7182) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&_stateVer_=cdcr-source:4&wt=javabin&version=2} hits=1000 status=0 QTime=1
[junit4] 2> 508756 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e36febf3d0006, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.base/java.lang.Thread.run(Thread.java:844)
[junit4] 2> 508761 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[B4D033E20BD53376]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 508763 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[B4D033E20BD53376]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:51714/solr ready
[junit4] 2> 508765 INFO (qtp32125918-7141) [ ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=cdcr-target&name=cdcr-target&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 508769 INFO (OverseerThreadFactory-2538-thread-1) [ ] o.a.s.c.CreateCollectionCmd Create collection cdcr-target
[junit4] 2> 508880 INFO (OverseerStateUpdate-98576709606768643-127.0.0.1:51718_solr-n_0000000000) [ ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"cdcr-target",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"cdcr-target_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:51718/solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 509081 INFO (qtp32125918-7143) [ ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=cdcr-target&newCollection=true&name=cdcr-target_shard1_replica_n1&action=CREATE&numShards=1&collection=cdcr-target&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 509082 INFO (qtp32125918-7143) [ ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 509189 INFO (zkCallback-2231-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 509189 INFO (zkCallback-2231-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 510103 INFO (qtp32125918-7143) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 510107 INFO (qtp32125918-7143) [ ] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema.xml instead
[junit4] 2> 510110 INFO (qtp32125918-7143) [ ] o.a.s.s.IndexSchema [cdcr-target_shard1_replica_n1] Schema name=minimal
[junit4] 2> 510112 INFO (qtp32125918-7143) [ ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 510113 INFO (qtp32125918-7143) [ ] o.a.s.s.ManagedIndexSchema Created and persisted managed schema znode at /configs/cdcr-target/managed-schema
[junit4] 2> 510116 INFO (qtp32125918-7143) [ ] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, renamed the non-managed schema /configs/cdcr-target/schema.xml to /configs/cdcr-target/schema.xml.bak
[junit4] 2> 510116 INFO (qtp32125918-7143) [ ] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-target_shard1_replica_n1' using configuration from collection cdcr-target, trusted=true
[junit4] 2> 510117 INFO (qtp32125918-7143) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51718.solr.core.cdcr-target.shard1.replica_n1' (registry 'solr.core.cdcr-target.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@135374d
[junit4] 2> 510117 INFO (qtp32125918-7143) [ ] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 510117 INFO (qtp32125918-7143) [ ] o.a.s.c.SolrCore [[cdcr-target_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrBootstrapTest_B4D033E20BD53376-001\cdcr-target-001\node1\cdcr-target_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrBootstrapTest_B4D033E20BD53376-001\cdcr-target-001\node1\.\cdcr-target_shard1_replica_n1\data\]
[junit4] 2> 510196 INFO (qtp32125918-7143) [ ] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 510196 INFO (qtp32125918-7143) [ ] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 510198 INFO (qtp32125918-7143) [ ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 510198 INFO (qtp32125918-7143) [ ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 510199 INFO (qtp32125918-7143) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@d7b21f[cdcr-target_shard1_replica_n1] main]
[junit4] 2> 510202 INFO (qtp32125918-7143) [ ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-target
[junit4] 2> 510202 INFO (qtp32125918-7143) [ ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-target
[junit4] 2> 510202 INFO (qtp32125918-7143) [ ] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-target/managed-schema
[junit4] 2> 510203 INFO (qtp32125918-7143) [ ] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 510204 INFO (qtp32125918-7143) [ ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 510207 INFO (qtp32125918-7143) [ ] o.a.s.h.CdcrBufferStateManager Created znode /collections/cdcr-target/cdcr/state/buffer
[junit4] 2> 510209 INFO (qtp32125918-7143) [ ] o.a.s.h.CdcrProcessStateManager Created znode /collections/cdcr-target/cdcr/state/process
[junit4] 2> 510210 INFO (searcherExecutor-2555-thread-1) [ ] o.a.s.c.SolrCore [cdcr-target_shard1_replica_n1] Registered new searcher Searcher@d7b21f[cdcr-target_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 510210 INFO (qtp32125918-7143) [ ] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577227359566692352
[junit4] 2> 510217 INFO (qtp32125918-7143) [ ] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 510217 INFO (qtp32125918-7143) [ ] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 510217 INFO (qtp32125918-7143) [ ] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:51718/solr/cdcr-target_shard1_replica_n1/
[junit4] 2> 510217 INFO (qtp32125918-7143) [ ] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 510217 INFO (qtp32125918-7143) [ ] o.a.s.c.SyncStrategy http://127.0.0.1:51718/solr/cdcr-target_shard1_replica_n1/ has no replicas
[junit4] 2> 510217 INFO (qtp32125918-7143) [ ] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 510220 INFO (zkCallback-2231-thread-2) [ ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-target:shard1
[junit4] 2> 510222 INFO (qtp32125918-7143) [ ] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:51718/solr/cdcr-target_shard1_replica_n1/ shard1
[junit4] 2> 510326 INFO (zkCallback-2231-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 510326 INFO (zkCallback-2231-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 510375 INFO (qtp32125918-7143) [ ] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 510377 INFO (qtp32125918-7143) [ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=cdcr-target&newCollection=true&name=cdcr-target_shard1_replica_n1&action=CREATE&numShards=1&collection=cdcr-target&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1295
[junit4] 2> 510380 INFO (qtp32125918-7141) [ ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 510482 INFO (zkCallback-2231-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 510482 INFO (zkCallback-2231-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 510771 INFO (OverseerCollectionConfigSetProcessor-98576709606768643-127.0.0.1:51718_solr-n_0000000000) [ ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 511380 INFO (qtp32125918-7141) [ ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=cdcr-target&name=cdcr-target&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=2615
[junit4] 2> 511384 INFO (zkCallback-2231-thread-1) [ ] o.a.s.h.CdcrProcessStateManager The CDCR process state has changed: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/cdcr/state/process @ cdcr-target:shard1
[junit4] 2> 511385 INFO (qtp32125918-7144) [ ] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&_stateVer_=cdcr-target:4&action=start&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 511385 INFO (zkCallback-2231-thread-1) [ ] o.a.s.h.CdcrProcessStateManager Received new CDCR process state from watcher: STARTED @ cdcr-target:shard1
[junit4] 2> 511391 INFO (qtp5308053-7185) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 511392 INFO (qtp5308053-7185) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:51714/solr ready
[junit4] 2> 511399 INFO (qtp32125918-7142) [ ] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
[junit4] 2> 511399 INFO (qtp32125918-7142) [ ] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 511399 INFO (qtp32125918-7146) [ ] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/cdcr params={_stateVer_=cdcr-target:4&action=COLLECTIONCHECKPOINT&wt=javabin&version=2} status=0 QTime=5
[junit4] 2> 511400 INFO (qtp5308053-7185) [ ] o.a.s.h.CdcrReplicatorManager Create new update log reader for target cdcr-target with checkpoint -1 @ cdcr-source:shard1
[junit4] 2> 511400 INFO (qtp5308053-7185) [ ] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-target, shard: shard1
[junit4] 2> 511401 INFO (qtp5308053-7185) [ ] o.a.s.h.CdcrReplicatorManager Submitting bootstrap task to executor
[junit4] 2> 511402 INFO (zkCallback-2243-thread-2) [ ] o.a.s.h.CdcrProcessStateManager The CDCR process state has changed: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/cdcr/state/process @ cdcr-source:shard1
[junit4] 2> 511402 INFO (qtp5308053-7185) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&_stateVer_=cdcr-source:4&action=start&wt=javabin&version=2} status=0 QTime=17
[junit4] 2> 511404 INFO (zkCallback-2243-thread-2) [ ] o.a.s.h.CdcrProcessStateManager Received new CDCR process state from watcher: STARTED @ cdcr-source:shard1
[junit4] 2> 511404 INFO (cdcr-bootstrap-status-2255-thread-1) [ ] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-target shard: shard1 leader: http://127.0.0.1:51718/solr/cdcr-target_shard1_replica_n1/
[junit4] 2> 511407 INFO (qtp5308053-7186) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&_stateVer_=cdcr-source:4&action=queues&wt=javabin&version=2} status=0 QTime=2
[junit4] 1> Cdcr queue response: {responseHeader={status=0,QTime=2},queues={127.0.0.1:51714/solr={cdcr-target={queueSize=1010,lastTimestamp=}}},tlogTotalSize=47659,tlogTotalCount=10,updateLogSynchronizer=stopped}
[junit4] 2> 511408 INFO (qtp32125918-7143) [ ] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&masterUrl=http://127.0.0.1:51742/solr/cdcr-source_shard1_replica_n1/&action=BOOTSTRAP&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 511408 INFO (qtp32125918-7144) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577227360822886400,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 511411 INFO (qtp32125918-7144) [ ] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 511413 INFO (qtp32125918-7144) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 511413 INFO (qtp32125918-7144) [ ] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-target:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=5
[junit4] 2> 511413 INFO (recoveryExecutor-2229-thread-1) [ ] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 511413 INFO (qtp32125918-7145) [ ] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&_stateVer_=cdcr-target:4&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 511416 INFO (qtp32125918-7139) [ ] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&action=BOOTSTRAP_STATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 511416 INFO (cdcr-bootstrap-status-2255-thread-1) [ ] o.a.s.h.CdcrReplicatorManager CDCR bootstrap running for 1 seconds, sleeping for 2000 ms
[junit4] 2> 511417 INFO (qtp5308053-7180) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577227360832323584,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 511417 INFO (qtp5308053-7180) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@e41152 commitCommandVersion:1577227360832323584
[junit4] 2> 511430 INFO (qtp5308053-7180) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@17e4ccd[cdcr-source_shard1_replica_n1] realtime]
[junit4] 2> 511430 INFO (qtp5308053-7180) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 511430 INFO (qtp5308053-7180) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&commit_end_point=true&wt=javabin&version=2} status=0 QTime=13
[junit4] 2> 511432 INFO (qtp5308053-7182) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 511432 INFO (recoveryExecutor-2229-thread-1) [ ] o.a.s.h.IndexFetcher Master's generation: 12
[junit4] 2> 511432 INFO (recoveryExecutor-2229-thread-1) [ ] o.a.s.h.IndexFetcher Master's version: 1504161225159
[junit4] 2> 511432 INFO (recoveryExecutor-2229-thread-1) [ ] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 511432 INFO (recoveryExecutor-2229-thread-1) [ ] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 511432 INFO (recoveryExecutor-2229-thread-1) [ ] o.a.s.h.IndexFetcher Starting replication process
[junit4] 2> 511441 INFO (qtp5308053-7185) [ ] o.a.s.h.ReplicationHandler Adding tlog files to list: [{size=4657, name=tlog.0000000000000000000.1577227357505191936}, {size=4778, name=tlog.0000000000000000001.1577227357547134976}, {size=4778, name=tlog.0000000000000000002.1577227357586980864}, {size=4778, name=tlog.0000000000000000003.1577227357631021056}, {size=4778, name=tlog.0000000000000000004.1577227357676109824}, {size=4778, name=tlog.0000000000000000005.1577227357735878656}, {size=4778, name=tlog.0000000000000000006.1577227357779918848}, {size=4778, name=tlog.0000000000000000007.1577227357837590528}, {size=4778, name=tlog.0000000000000000008.1577227357893165056}, {size=4778, name=tlog.0000000000000000009.1577227357963419648}]
[junit4] 2> 511441 INFO (qtp5308053-7185) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&wt=javabin&version=2&command=filelist} status=0 QTime=7
[junit4] 2> 511442 INFO (recoveryExecutor-2229-thread-1) [ ] o.a.s.h.IndexFetcher Number of files in latest index in master: 13
[junit4] 2> 511442 INFO (recoveryExecutor-2229-thread-1) [ ] o.a.s.h.IndexFetcher Number of tlog files in master: 10
[junit4] 2> 511443 INFO (recoveryExecutor-2229-thread-1) [ ] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 511443 INFO (recoveryExecutor-2229-thread-1) [ ] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MockDirectoryWrapper(RAMDirectory@161158 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@12ad631)
[junit4] 2> 511443 INFO (qtp5308053-7186) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_Asserting_0.dvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 511446 INFO (qtp5308053-7187) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 511448 INFO (qtp5308053-7180) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_Asserting_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 511450 INFO (qtp5308053-7184) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 511450 INFO (qtp5308053-7183) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 511452 INFO (qtp5308053-7182) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_Asserting_0.dvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 511453 INFO (qtp5308053-7185) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_Asserting_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 511454 INFO (qtp5308053-7186) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_Asserting_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 511456 INFO (qtp5308053-7187) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_Asserting_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 511457 INFO (qtp5308053-7180) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.nvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 511459 INFO (qtp5308053-7184) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.fnm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 511460 INFO (qtp5308053-7183) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 511461 INFO (qtp5308053-7182) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=segments_c&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 511462 INFO (recoveryExecutor-2229-thread-1) [ ] o.a.s.h.IndexFetcher Starting download of tlog files from master: [{size=4657, name=tlog.0000000000000000000.1577227357505191936}, {size=4778, name=tlog.0000000000000000001.1577227357547134976}, {size=4778, name=tlog.0000000000000000002.1577227357586980864}, {size=4778, name=tlog.0000000000000000003.1577227357631021056}, {size=4778, name=tlog.0000000000000000004.1577227357676109824}, {size=4778, name=tlog.0000000000000000005.1577227357735878656}, {size=4778, name=tlog.0000000000000000006.1577227357779918848}, {size=4778, name=tlog.0000000000000000007.1577227357837590528}, {size=4778, name=tlog.0000000000000000008.1577227357893165056}, {size=4778, name=tlog.0000000000000000009.1577227357963419648}]
[junit4] 2> 511463 INFO (qtp5308053-7185) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000000.1577227357505191936&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 511466 INFO (qtp5308053-7186) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000001.1577227357547134976&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 511469 INFO (qtp5308053-7187) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000002.1577227357586980864&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 511479 INFO (qtp5308053-7180) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000003.1577227357631021056&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 511481 INFO (qtp5308053-7184) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000004.1577227357676109824&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 511483 INFO (qtp5308053-7183) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000005.1577227357735878656&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 511485 INFO (qtp5308053-7182) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000006.1577227357779918848&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 511488 INFO (qtp5308053-7185) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000007.1577227357837590528&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 511490 INFO (qtp5308053-7186) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000008.1577227357893165056&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 511492 INFO (qtp5308053-7187) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000009.1577227357963419648&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 511493 INFO (recoveryExecutor-2229-thread-1) [ ] o.a.s.h.IndexFetcher Total time taken for download (fullCopy=false,bytesDownloaded=96726) : 0 secs (null bytes/sec) to MockDirectoryWrapper(RAMDirectory@161158 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@12ad631)
[junit4] 2> 511503 INFO (recoveryExecutor-2229-thread-1) [ ] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 511503 INFO (recoveryExecutor-2229-thread-1) [ ] o.a.s.h.IndexFetcher Reloading SolrCore cdcr-target_shard1_replica_n1
[junit4] 2> 511513 INFO (Thread-1441) [ ] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 511519 INFO (Thread-1441) [ ] o.a.s.s.IndexSchema [cdcr-target_shard1_replica_n1] Schema name=minimal
[junit4] 2> 511521 INFO (Thread-1441) [ ] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 511521 INFO (Thread-1441) [ ] o.a.s.c.CoreContainer Reloading SolrCore 'cdcr-target_shard1_replica_n1' using configuration from collection cdcr-target
[junit4] 2> 511546 INFO (Thread-1441) [ ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51718.solr.core.cdcr-target.shard1.replica_n1' (registry 'solr.core.cdcr-target.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@135374d
[junit4] 2> 511546 INFO (Thread-1441) [ ] o.a.s.c.SolrCore [[cdcr-target_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrBootstrapTest_B4D033E20BD53376-001\cdcr-target-001\node1\cdcr-target_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrBootstrapTest_B4D033E20BD53376-001\cdcr-target-001\node1\.\cdcr-target_shard1_replica_n1\data\]
[junit4] 2> 511645 INFO (Thread-1441) [ ] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 511645 INFO (Thread-1441) [ ] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 511648 INFO (Thread-1441) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@bfc948[cdcr-target_shard1_replica_n1] main]
[junit4] 2> 511649 INFO (Thread-1441) [ ] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-target
[junit4] 2> 511650 INFO (Thread-1441) [ ] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-target
[junit4] 2> 511650 INFO (Thread-1441) [ ] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-target/managed-schema
[junit4] 2> 511652 INFO (Thread-1441) [ ] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 511652 INFO (Thread-1441) [ ] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 511659 INFO (searcherExecutor-2567-thread-1) [ ] o.a.s.c.SolrCore [cdcr-target_shard1_replica_n1] Registered new searcher Searcher@bfc948[cdcr-target_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_a(8.0.0):C1000)))}
[junit4] 2> 511667 INFO (Thread-1441) [ ] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 511668 INFO (Thread-1441) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1be493b[cdcr-target_shard1_replica_n1] main]
[junit4] 2> 511674 INFO (searcherExecutor-2567-thread-1) [ ] o.a.s.c.SolrCore [cdcr-target_shard1_replica_n1] Registered new searcher Searcher@1be493b[cdcr-target_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_a(8.0.0):C1000)))}
[junit4] 2> 511674 INFO (Thread-1441) [ ] o.a.s.c.SolrCore [cdcr-target_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@38a863
[junit4] 2> 511675 INFO (Thread-1441) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.cdcr-target.shard1.replica_n1, tag=3713123
[junit4] 2> 511675 INFO (Thread-1441) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@12eb8d7: rootName = solr_51718, domain = solr.core.cdcr-target.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.cdcr-target.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@12437cd
[junit4] 2> 511686 INFO (Thread-1441) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.cdcr-target.shard1.leader, tag=3713123
[junit4] 2> 511686 INFO (Thread-1441) [ ] o.a.s.h.CdcrRequestHandler Solr core is being closed - shutting down CDCR handler @ cdcr-target:shard1
[junit4] 2> 511688 INFO (recoveryExecutor-2229-thread-1) [ ] o.a.s.h.CdcrRequestHandler No replay needed.
[junit4] 2> 512416 INFO (qtp32125918-7142) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577227361879851008,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 512416 INFO (qtp32125918-7142) [ ] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 512416 INFO (qtp32125918-7142) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 512417 INFO (qtp32125918-7142) [ ] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-target:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 512418 INFO (qtp32125918-7146) [ ] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&_stateVer_=cdcr-target:4&wt=javabin&version=2} hits=1000 status=0 QTime=0
[junit4] 2> 512420 INFO (qtp32125918-7141) [ ] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&action=CANCEL_BOOTSTRAP&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 513416 WARN (cdcr-bootstrap-status-2255-thread-1) [ ] o.a.s.h.CdcrReplicatorManager Cancelling waiting for bootstrap on target: cdcr-target shard: shard1 to complete
[junit4] 2> 513418 INFO (zkCallback-2243-thread-2) [ ] o.a.s.h.CdcrProcessStateManager The CDCR process state has changed: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/cdcr/state/process @ cdcr-source:shard1
[junit4] 2> 513418 INFO (qtp5308053-7180) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&_stateVer_=cdcr-source:4&action=stop&wt=javabin&version=2} status=0 QTime=998
[junit4] 2> 513418 INFO (zkCallback-2243-thread-2) [ ] o.a.s.h.CdcrProcessStateManager Received new CDCR process state from watcher: STOPPED @ cdcr-source:shard1
[junit4] 2> 513419 INFO (zkCallback-2243-thread-2) [ ] o.a.s.h.CdcrBufferStateManager The CDCR buffer state has changed: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/cdcr/state/buffer @ cdcr-source:shard1
[junit4] 2> 513419 INFO (qtp5308053-7184) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&_stateVer_=cdcr-source:4&action=disablebuffer&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 513419 INFO (zkCallback-2243-thread-2) [ ] o.a.s.h.CdcrBufferStateManager Received new CDCR buffer state from watcher: DISABLED @ cdcr-source:shard1
[junit4] 1> Adding 100 docs with commit=true, numDocs=1100
[junit4] 2> 513427 INFO (qtp5308053-7183) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=6
[junit4] 2> 513428 INFO (qtp5308053-7182) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577227362941009920,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 513428 INFO (qtp5308053-7182) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@e41152 commitCommandVersion:1577227362941009920
[junit4] 2> 513452 INFO (qtp5308053-7182) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@c28609[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 513455 INFO (searcherExecutor-2549-thread-1) [ ] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@c28609[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_a(8.0.0):C1000) Uninverting(_b(8.0.0):C100)))}
[junit4] 2> 513456 INFO (qtp5308053-7182) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 513494 INFO (qtp5308053-7182) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=67
[junit4] 1> Adding 100 docs with commit=true, numDocs=1200
[junit4] 2> 513501 INFO (qtp5308053-7185) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=5
[junit4] 2> 513502 INFO (qtp5308053-7186) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577227363018604544,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 513502 INFO (qtp5308053-7186) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@e41152 commitCommandVersion:1577227363018604544
[junit4] 2> 513527 INFO (qtp5308053-7186) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@161a8cd[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 513528 INFO (qtp5308053-7186) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 513529 INFO (searcherExecutor-2549-thread-1) [ ] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@161a8cd[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_a(8.0.0):C1000) Uninverting(_b(8.0.0):C100) Uninverting(_c(8.0.0):C100)))}
[junit4] 2> 513530 INFO (qtp5308053-7186) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=27
[junit4] 1> Adding 100 docs with commit=true, numDocs=1300
[junit4] 2> 513536 INFO (qtp5308053-7187) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=5
[junit4] 2> 513536 INFO (qtp5308053-7180) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577227363054256128,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 513536 INFO (qtp5308053-7180) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@e41152 commitCommandVersion:1577227363054256128
[junit4] 2> 513569 INFO (qtp5308053-7180) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@a30ba4[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 513570 INFO (qtp5308053-7180) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 513571 INFO (searcherExecutor-2549-thread-1) [ ] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@a30ba4[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_a(8.0.0):C1000) Uninverting(_b(8.0.0):C100) Uninverting(_c(8.0.0):C100) Uninverting(_d(8.0.0):C100)))}
[junit4] 2> 513572 INFO (qtp5308053-7180) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=35
[junit4] 1> Adding 100 docs with commit=true, numDocs=1400
[junit4] 2> 513578 INFO (qtp5308053-7184) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=5
[junit4] 2> 513579 INFO (qtp5308053-7183) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577227363099344896,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 513579 INFO (qtp5308053-7183) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@e41152 commitCommandVersion:1577227363099344896
[junit4] 2> 513609 INFO (qtp5308053-7183) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@111e7ae[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 513610 INFO (qtp5308053-7183) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 513611 INFO (searcherExecutor-2549-thread-1) [ ] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@111e7ae[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_a(8.0.0):C1000) Uninverting(_b(8.0.0):C100) Uninverting(_c(8.0.0):C100) Uninverting(_d(8.0.0):C100) Uninverting(_e(8.0.0):C100)))}
[junit4] 2> 513611 INFO (qtp5308053-7183) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=32
[junit4] 1> Adding 100 docs with commit=true, numDocs=1500
[junit4] 2> 513616 INFO (qtp5308053-7182) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=4
[junit4] 2> 513617 INFO (qtp5308053-7185) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577227363139190784,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 513617 INFO (qtp5308053-7185) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@e41152 commitCommandVersion:1577227363139190784
[junit4] 2> 513657 INFO (qtp5308053-7185) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1a4c728[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 513658 INFO (qtp5308053-7185) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 513659 INFO (searcherExecutor-2549-thread-1) [ ] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@1a4c728[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_a(8.0.0):C1000) Uninverting(_b(8.0.0):C100) Uninverting(_c(8.0.0):C100) Uninverting(_d(8.0.0):C100) Uninverting(_e(8.0.0):C100) Uninverting(_f(8.0.0):C100)))}
[junit4] 2> 513659 INFO (qtp5308053-7185) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=41
[junit4] 1> Adding 100 docs with commit=true, numDocs=1600
[junit4] 2> 513664 INFO (qtp5308053-7186) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=4
[junit4] 2> 513665 INFO (qtp5308053-7187) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577227363189522432,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 513665 INFO (qtp5308053-7187) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@e41152 commitCommandVersion:1577227363189522432
[junit4] 2> 513712 INFO (qtp5308053-7187) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@18e4a7f[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 513713 INFO (qtp5308053-7187) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 513714 INFO (searcherExecutor-2549-thread-1) [ ] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@18e4a7f[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_a(8.0.0):C1000) Uninverting(_b(8.0.0):C100) Uninverting(_c(8.0.0):C100) Uninverting(_d(8.0.0):C100) Uninverting(_e(8.0.0):C100) Uninverting(_f(8.0.0):C100) Uninverting(_g(8.0.0):C100)))}
[junit4] 2> 513714 INFO (qtp5308053-7187) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=49
[junit4] 1> Adding 100 docs with commit=true, numDocs=1700
[junit4] 2> 513719 INFO (qtp5308053-7180) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=4
[junit4] 2> 513721 INFO (qtp5308053-7184) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577227363247194112,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 513721 INFO (qtp5308053-7184) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@e41152 commitCommandVersion:1577227363247194112
[junit4] 2> 513760 INFO (qtp5308053-7184) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@8e6e6d[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 513761 INFO (qtp5308053-7184) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 513762 INFO (searcherExecutor-2549-thread-1) [ ] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@8e6e6d[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_a(8.0.0):C1000) Uninverting(_b(8.0.0):C100) Uninverting(_c(8.0.0):C100) Uninverting(_d(8.0.0):C100) Uninverting(_e(8.0.0):C100) Uninverting(_f(8.0.0):C100) Uninverting(_g(8.0.0):C100) Uninverting(_h(8.0.0):C100)))}
[junit4] 2> 513762 INFO (qtp5308053-7184) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=42
[junit4] 1> Adding 100 docs with commit=true, numDocs=1800
[junit4] 2> 513768 INFO (qtp5308053-7183) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=4
[junit4] 2> 513769 INFO (qtp5308053-7182) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577227363298574336,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 513769 INFO (qtp5308053-7182) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@e41152 commitCommandVersion:1577227363298574336
[junit4] 2> 513812 INFO (qtp5308053-7182) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@48ddf9[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 513814 INFO (qtp5308053-7182) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 513815 INFO (searcherExecutor-2549-thread-1) [ ] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@48ddf9[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_a(8.0.0):C1000) Uninverting(_b(8.0.0):C100) Uninverting(_c(8.0.0):C100) Uninverting(_d(8.0.0):C100) Uninverting(_e(8.0.0):C100) Uninverting(_f(8.0.0):C100) Uninverting(_g(8.0.0):C100) Uninverting(_h(8.0.0):C100) Uninverting(_i(8.0.0):C100)))}
[junit4] 2> 513815 INFO (qtp5308053-7182) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=46
[junit4] 1> Adding 100 docs with commit=true, numDocs=1900
[junit4] 2> 513820 INFO (qtp5308053-7185) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=4
[junit4] 2> 513820 INFO (qtp5308053-7186) [ ] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577227363352051722,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 513820 INFO (qtp5308053-7186) [ ] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@e41152 commitCommandVersion:1577227363352051722
[junit4] 2> 513869 INFO (qtp5308053-7186) [ ] o.a.s.s.SolrIndexSearcher Opening [Searcher@1289991[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 513870 INFO (qtp5308053-7186) [ ] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 513872 INFO (searcherExecutor-2549-thread-1) [ ] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@1289991[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_a(8.0.0):C1000) Uninverting(_b(8.0.0):C100) Uninverting(_c(8.0.0):C100) Uninverting(_d(8.0.0):C100) Uninverting(_e(8.0.0):C100) Uninverting(_f(8.0.0):C100) Uninverting(_g(8.0.0):C100) Uninverting(_h(8.0.0):C100) Uninverting(_i(8.0.0):C100) Uninverting(_j(8.0.0):C100)))}
[junit4] 2> 513873 INFO (qtp5308053-7186) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source
[...truncated too long message...]
4] 2> 3 /solr/aliases.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2> 2 /solr/configs/cdcr-source
[junit4] 2> 2 /solr/security.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 10 /solr/collections/cdcr-source/state.json
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 2 /solr/collections/cdcr-source/cdcr/state/process
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/collections
[junit4] 2> 2 /solr/overseer/queue
[junit4] 2> 2 /solr/live_nodes
[junit4] 2> 2 /solr/overseer/collection-queue-work
[junit4] 2>
[junit4] 2> 664235 INFO (jetty-closer-2294-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@f8e22d{HTTP/1.1,[http/1.1]}{127.0.0.1:?}
[junit4] 2> 664236 INFO (jetty-closer-2294-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=12838399
[junit4] 2> 664236 INFO (jetty-closer-2294-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 664236 INFO (jetty-closer-2294-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@19a8f34: rootName = solr_52106, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@ea7479
[junit4] 2> 664248 INFO (jetty-closer-2294-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 664248 INFO (jetty-closer-2294-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@cf41cd: rootName = solr_52106, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@7d96b6
[junit4] 2> 664254 INFO (jetty-closer-2294-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 664254 INFO (jetty-closer-2294-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1656dda: rootName = solr_52106, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@18ccb05
[junit4] 2> 664257 INFO (jetty-closer-2294-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 664258 INFO (coreCloseExecutor-2689-thread-1) [ ] o.a.s.c.SolrCore [cdcr-target_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@103b172
[junit4] 2> 664258 INFO (coreCloseExecutor-2689-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.cdcr-target.shard1.replica_n1, tag=17019250
[junit4] 2> 664258 INFO (coreCloseExecutor-2689-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@11969cd: rootName = solr_52106, domain = solr.core.cdcr-target.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.cdcr-target.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@1a28c07
[junit4] 2> 664261 INFO (zkCallback-2300-thread-1) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 664261 INFO (zkCallback-2300-thread-2) [ ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 664271 INFO (coreCloseExecutor-2689-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.cdcr-target.shard1.leader, tag=17019250
[junit4] 2> 664271 INFO (coreCloseExecutor-2689-thread-1) [ ] o.a.s.h.CdcrRequestHandler Solr core is being closed - shutting down CDCR handler @ cdcr-target:shard1
[junit4] 2> 664271 INFO (jetty-closer-2294-thread-1) [ ] o.a.s.c.Overseer Overseer (id=98576719214673923-127.0.0.1:52106_solr-n_0000000000) closing
[junit4] 2> 664273 INFO (OverseerStateUpdate-98576719214673923-127.0.0.1:52106_solr-n_0000000000) [ ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:52106_solr
[junit4] 2> 664274 WARN (zkCallback-2300-thread-3) [ ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 664275 INFO (jetty-closer-2294-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1c373d4{/solr,null,UNAVAILABLE}
[junit4] 2> 664276 ERROR (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[B4D033E20BD53376]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 664276 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[B4D033E20BD53376]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:52102 52102
[junit4] 2> 664276 INFO (Thread-1474) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:52102 52102
[junit4] 2> 664277 WARN (Thread-1474) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2> 2 /solr/configs/cdcr-target/managed-schema
[junit4] 2> 2 /solr/collections/cdcr-target/leaders/shard1/leader
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/collections/cdcr-target/state.json
[junit4] 2> 3 /solr/collections/cdcr-target/cdcr/state/process
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 2 /solr/collections/cdcr-target/cdcr/state/buffer
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/live_nodes
[junit4] 2> 3 /solr/collections
[junit4] 2>
[junit4] 2> 664277 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[B4D033E20BD53376]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testConvertClusterToCdcrAndBootstrap
[junit4] 2> NOTE: leaving temporary files on disk at: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrBootstrapTest_B4D033E20BD53376-001
[junit4] 2> Aug 31, 2017 6:36:18 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 5 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70), sim=RandomSimilarity(queryNorm=true): {}, locale=ne-IN, timezone=Asia/Seoul
[junit4] 2> NOTE: Windows 10 10.0 x86/Oracle Corporation 9 (32-bit)/cpus=3,threads=1,free=200444888,total=299368448
[junit4] 2> NOTE: All tests run in this JVM: [UpdateLogTest, TestSubQueryTransformerCrossCore, TestSolrCloudWithSecureImpersonation, TestAtomicUpdateErrorCases, TestFieldSortValues, CleanupOldIndexTest, BlockDirectoryTest, TestFoldingMultitermQuery, CoreAdminRequestStatusTest, CursorPagingTest, CollectionTooManyReplicasTest, DistributedSpellCheckComponentTest, ConfigSetsAPITest, BinaryUpdateRequestHandlerTest, DataDrivenBlockJoinTest, TolerantUpdateProcessorTest, DateFieldTest, AnalyticsMergeStrategyTest, TestAuthenticationFramework, RAMDirectoryFactoryTest, AddSchemaFieldsUpdateProcessorFactoryTest, HdfsNNFailoverTest, SuggestComponentContextFilterQueryTest, TestHashPartitioner, TestRestManager, TestFuzzyAnalyzedSuggestions, BasicAuthStandaloneTest, TestSQLHandler, TestPseudoReturnFields, ZkControllerTest, HLLUtilTest, TestFieldCacheSortRandom, DateRangeFieldTest, TestJsonFacetRefinement, CachingDirectoryFactoryTest, ChangedSchemaMergeTest, TestUniqueKeyFieldResource, TestCopyFieldCollectionResource, SortByFunctionTest, TestReloadDeadlock, TestReqParamsAPI, TestCollapseQParserPlugin, SolrTestCaseJ4Test, OverriddenZkACLAndCredentialsProvidersTest, PeerSyncReplicationTest, HdfsSyncSliceTest, BlockJoinFacetSimpleTest, TestDFISimilarityFactory, CreateCollectionCleanupTest, TestSolrCloudSnapshots, TestOverriddenPrefixQueryForCustomFieldType, TestCursorMarkWithoutUniqueKey, TestLeaderElectionWithEmptyReplica, TestConfigsApi, TestReplicationHandler, TestLeaderInitiatedRecoveryThread, HdfsBasicDistributedZkTest, TestNumericTerms32, MoveReplicaTest, TestPhraseSuggestions, TestSSLRandomization, TestSweetSpotSimilarityFactory, TestHighlightDedupGrouping, ConjunctionSolrSpellCheckerTest, DistributedVersionInfoTest, TestCoreContainer, TestInitQParser, TestUtils, WordBreakSolrSpellCheckerTest, CdcrBootstrapTest]
[junit4] Completed [153/732 (1!)] on J0 in 160.69s, 3 tests, 1 failure <<< FAILURES!
[...truncated 40999 lines...]
BUILD FAILED
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\build.xml:810: The following error occurred while executing this line:
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\build.xml:690: The following error occurred while executing this line:
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\build.xml:678: Source checkout is modified!!! Offending files:
* solr/licenses/httpmime-4.5.3.jar.sha1
Total time: 76 minutes 11 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any
[JENKINS] Lucene-Solr-master-Windows (32bit/jdk1.8.0_144) - Build #
6860 - Still Failing!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-master-Windows/6860/
Java: 32bit/jdk1.8.0_144 -client -XX:+UseSerialGC
1 tests failed.
FAILED: org.apache.solr.cloud.TestCloudRecovery.corruptedLogTest
Error Message:
Timeout waiting for all live and active
Stack Trace:
java.lang.AssertionError: Timeout waiting for all live and active
at __randomizedtesting.SeedInfo.seed([4EE4C0B87CE14867:CD929F4AAA9846C6]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.assertTrue(Assert.java:43)
at org.apache.solr.cloud.TestCloudRecovery.corruptedLogTest(TestCloudRecovery.java:184)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12903 lines...]
[junit4] Suite: org.apache.solr.cloud.TestCloudRecovery
[junit4] 2> 2745491 INFO (SUITE-TestCloudRecovery-seed#[4EE4C0B87CE14867]-worker) [ ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
[junit4] 2> Creating dataDir: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_4EE4C0B87CE14867-001\init-core-data-001
[junit4] 2> 2745493 WARN (SUITE-TestCloudRecovery-seed#[4EE4C0B87CE14867]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=16 numCloses=16
[junit4] 2> 2745494 INFO (SUITE-TestCloudRecovery-seed#[4EE4C0B87CE14867]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 2745499 INFO (SUITE-TestCloudRecovery-seed#[4EE4C0B87CE14867]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 2745503 INFO (SUITE-TestCloudRecovery-seed#[4EE4C0B87CE14867]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_4EE4C0B87CE14867-001\tempDir-001
[junit4] 2> 2745503 INFO (SUITE-TestCloudRecovery-seed#[4EE4C0B87CE14867]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2745504 INFO (Thread-4702) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2745504 INFO (Thread-4702) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2745510 ERROR (Thread-4702) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2745607 INFO (SUITE-TestCloudRecovery-seed#[4EE4C0B87CE14867]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:51292
[junit4] 2> 2745630 INFO (jetty-launcher-4753-thread-1) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2745630 INFO (jetty-launcher-4753-thread-2) [ ] o.e.j.s.Server jetty-9.3.20.v20170531
[junit4] 2> 2745632 INFO (jetty-launcher-4753-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1184358{/solr,null,AVAILABLE}
[junit4] 2> 2745632 INFO (jetty-launcher-4753-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1658ea4{/solr,null,AVAILABLE}
[junit4] 2> 2745641 INFO (jetty-launcher-4753-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1e953ac{SSL,[ssl, http/1.1]}{127.0.0.1:51296}
[junit4] 2> 2745641 INFO (jetty-launcher-4753-thread-1) [ ] o.e.j.s.Server Started @2754096ms
[junit4] 2> 2745641 INFO (jetty-launcher-4753-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@16b1ab0{SSL,[ssl, http/1.1]}{127.0.0.1:51297}
[junit4] 2> 2745641 INFO (jetty-launcher-4753-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51296}
[junit4] 2> 2745641 INFO (jetty-launcher-4753-thread-2) [ ] o.e.j.s.Server Started @2754096ms
[junit4] 2> 2745641 INFO (jetty-launcher-4753-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=51297}
[junit4] 2> 2745641 ERROR (jetty-launcher-4753-thread-1) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2745641 ERROR (jetty-launcher-4753-thread-2) [ ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2745641 INFO (jetty-launcher-4753-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2745642 INFO (jetty-launcher-4753-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 8.0.0
[junit4] 2> 2745642 INFO (jetty-launcher-4753-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2745642 INFO (jetty-launcher-4753-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2745642 INFO (jetty-launcher-4753-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 2745642 INFO (jetty-launcher-4753-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 2745642 INFO (jetty-launcher-4753-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-31T18:10:46.606Z
[junit4] 2> 2745642 INFO (jetty-launcher-4753-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-31T18:10:46.606Z
[junit4] 2> 2745649 INFO (jetty-launcher-4753-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2745650 INFO (jetty-launcher-4753-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2745669 INFO (jetty-launcher-4753-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51292/solr
[junit4] 2> 2745671 INFO (jetty-launcher-4753-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:51292/solr
[junit4] 2> 2745834 INFO (jetty-launcher-4753-thread-2) [n:127.0.0.1:51297_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2745835 INFO (jetty-launcher-4753-thread-2) [n:127.0.0.1:51297_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:51297_solr
[junit4] 2> 2745836 INFO (jetty-launcher-4753-thread-2) [n:127.0.0.1:51297_solr ] o.a.s.c.Overseer Overseer (id=98579450853654533-127.0.0.1:51297_solr-n_0000000000) starting
[junit4] 2> 2745838 INFO (jetty-launcher-4753-thread-1) [n:127.0.0.1:51296_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2745840 INFO (jetty-launcher-4753-thread-1) [n:127.0.0.1:51296_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51296_solr
[junit4] 2> 2745841 INFO (zkCallback-4765-thread-1-processing-n:127.0.0.1:51296_solr) [n:127.0.0.1:51296_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2745841 INFO (zkCallback-4764-thread-1-processing-n:127.0.0.1:51297_solr) [n:127.0.0.1:51297_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2745846 INFO (jetty-launcher-4753-thread-2) [n:127.0.0.1:51297_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:51297_solr
[junit4] 2> 2745848 INFO (zkCallback-4764-thread-1-processing-n:127.0.0.1:51297_solr) [n:127.0.0.1:51297_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2745848 INFO (zkCallback-4765-thread-1-processing-n:127.0.0.1:51296_solr) [n:127.0.0.1:51296_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 2746090 INFO (jetty-launcher-4753-thread-1) [n:127.0.0.1:51296_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51296.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@185ffcd
[junit4] 2> 2746104 INFO (jetty-launcher-4753-thread-1) [n:127.0.0.1:51296_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51296.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@185ffcd
[junit4] 2> 2746104 INFO (jetty-launcher-4753-thread-1) [n:127.0.0.1:51296_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51296.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@185ffcd
[junit4] 2> 2746106 INFO (jetty-launcher-4753-thread-1) [n:127.0.0.1:51296_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_4EE4C0B87CE14867-001\tempDir-001\node1\.
[junit4] 2> 2746188 INFO (jetty-launcher-4753-thread-2) [n:127.0.0.1:51297_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51297.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@185ffcd
[junit4] 2> 2746198 INFO (jetty-launcher-4753-thread-2) [n:127.0.0.1:51297_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51297.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@185ffcd
[junit4] 2> 2746198 INFO (jetty-launcher-4753-thread-2) [n:127.0.0.1:51297_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51297.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@185ffcd
[junit4] 2> 2746201 INFO (jetty-launcher-4753-thread-2) [n:127.0.0.1:51297_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_4EE4C0B87CE14867-001\tempDir-001\node2\.
[junit4] 2> 2746272 INFO (SUITE-TestCloudRecovery-seed#[4EE4C0B87CE14867]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 2746275 INFO (SUITE-TestCloudRecovery-seed#[4EE4C0B87CE14867]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:51292/solr ready
[junit4] 2> 2746338 INFO (qtp5470645-23737) [n:127.0.0.1:51297_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=0&collection.configName=config&maxShardsPerNode=2&name=collection1&nrtReplicas=0&action=CREATE&numShards=2&tlogReplicas=2&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2746342 INFO (OverseerThreadFactory-9099-thread-1-processing-n:127.0.0.1:51297_solr) [n:127.0.0.1:51297_solr ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 2746460 INFO (OverseerStateUpdate-98579450853654533-127.0.0.1:51297_solr-n_0000000000) [n:127.0.0.1:51297_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1_shard1_replica_t1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:51297/solr",
[junit4] 2> "type":"TLOG"}
[junit4] 2> 2746465 INFO (OverseerStateUpdate-98579450853654533-127.0.0.1:51297_solr-n_0000000000) [n:127.0.0.1:51297_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1_shard1_replica_t2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:51296/solr",
[junit4] 2> "type":"TLOG"}
[junit4] 2> 2746468 INFO (OverseerStateUpdate-98579450853654533-127.0.0.1:51297_solr-n_0000000000) [n:127.0.0.1:51297_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"collection1_shard2_replica_t3",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:51297/solr",
[junit4] 2> "type":"TLOG"}
[junit4] 2> 2746475 INFO (OverseerStateUpdate-98579450853654533-127.0.0.1:51297_solr-n_0000000000) [n:127.0.0.1:51297_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"collection1_shard2_replica_t5",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"https://127.0.0.1:51296/solr",
[junit4] 2> "type":"TLOG"}
[junit4] 2> 2746691 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=config&newCollection=true&name=collection1_shard1_replica_t1&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 2746693 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2746693 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=config&newCollection=true&name=collection1_shard2_replica_t3&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 2746735 INFO (qtp14482372-23745) [n:127.0.0.1:51296_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=config&newCollection=true&name=collection1_shard2_replica_t5&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 2746737 INFO (qtp14482372-23745) [n:127.0.0.1:51296_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2746769 INFO (qtp14482372-23746) [n:127.0.0.1:51296_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node6&collection.configName=config&newCollection=true&name=collection1_shard1_replica_t2&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG
[junit4] 2> 2746879 INFO (zkCallback-4764-thread-1-processing-n:127.0.0.1:51297_solr) [n:127.0.0.1:51297_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2746879 INFO (zkCallback-4765-thread-1-processing-n:127.0.0.1:51296_solr) [n:127.0.0.1:51296_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2746883 INFO (zkCallback-4764-thread-2-processing-n:127.0.0.1:51297_solr) [n:127.0.0.1:51297_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2746884 INFO (zkCallback-4765-thread-2-processing-n:127.0.0.1:51296_solr) [n:127.0.0.1:51296_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2747738 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2747738 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2747750 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.s.IndexSchema [collection1_shard2_replica_t3] Schema name=minimal
[junit4] 2> 2747753 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.s.IndexSchema [collection1_shard1_replica_t1] Schema name=minimal
[junit4] 2> 2747756 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2747756 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_t3' using configuration from collection collection1, trusted=true
[junit4] 2> 2747757 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2747757 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_t1' using configuration from collection collection1, trusted=true
[junit4] 2> 2747757 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51297.solr.core.collection1.shard2.replica_t3' (registry 'solr.core.collection1.shard2.replica_t3') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@185ffcd
[junit4] 2> 2747758 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51297.solr.core.collection1.shard1.replica_t1' (registry 'solr.core.collection1.shard1.replica_t1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@185ffcd
[junit4] 2> 2747758 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2747758 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2747758 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.SolrCore [[collection1_shard2_replica_t3] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_4EE4C0B87CE14867-001\tempDir-001\node2\collection1_shard2_replica_t3], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_4EE4C0B87CE14867-001\tempDir-001\node2\.\collection1_shard2_replica_t3\data\]
[junit4] 2> 2747758 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.SolrCore [[collection1_shard1_replica_t1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_4EE4C0B87CE14867-001\tempDir-001\node2\collection1_shard1_replica_t1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_4EE4C0B87CE14867-001\tempDir-001\node2\.\collection1_shard1_replica_t1\data\]
[junit4] 2> 2748037 INFO (qtp14482372-23745) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2748055 INFO (qtp14482372-23745) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.s.IndexSchema [collection1_shard2_replica_t5] Schema name=minimal
[junit4] 2> 2748140 INFO (qtp14482372-23745) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2748141 INFO (qtp14482372-23745) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_t5' using configuration from collection collection1, trusted=true
[junit4] 2> 2748147 INFO (qtp14482372-23745) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51296.solr.core.collection1.shard2.replica_t5' (registry 'solr.core.collection1.shard2.replica_t5') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@185ffcd
[junit4] 2> 2748147 INFO (qtp14482372-23745) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2748148 INFO (qtp14482372-23745) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.c.SolrCore [[collection1_shard2_replica_t5] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_4EE4C0B87CE14867-001\tempDir-001\node1\collection1_shard2_replica_t5], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_4EE4C0B87CE14867-001\tempDir-001\node1\.\collection1_shard2_replica_t5\data\]
[junit4] 2> 2748151 INFO (qtp14482372-23746) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 8.0.0
[junit4] 2> 2748239 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2748239 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2748244 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2748245 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2748249 INFO (qtp14482372-23746) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.s.IndexSchema [collection1_shard1_replica_t2] Schema name=minimal
[junit4] 2> 2748256 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.s.SolrIndexSearcher Opening [Searcher@13fb019[collection1_shard1_replica_t1] main]
[junit4] 2> 2748257 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
[junit4] 2> 2748258 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
[junit4] 2> 2748260 INFO (qtp14482372-23746) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2748260 INFO (qtp14482372-23746) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_t2' using configuration from collection collection1, trusted=true
[junit4] 2> 2748261 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2748261 INFO (qtp14482372-23746) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_51296.solr.core.collection1.shard1.replica_t2' (registry 'solr.core.collection1.shard1.replica_t2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@185ffcd
[junit4] 2> 2748261 INFO (qtp14482372-23746) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2748261 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577271216544153600
[junit4] 2> 2748262 INFO (qtp14482372-23746) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.c.SolrCore [[collection1_shard1_replica_t2] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_4EE4C0B87CE14867-001\tempDir-001\node1\collection1_shard1_replica_t2], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_4EE4C0B87CE14867-001\tempDir-001\node1\.\collection1_shard1_replica_t2\data\]
[junit4] 2> 2748263 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2748263 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2748264 INFO (searcherExecutor-9105-thread-1-processing-n:127.0.0.1:51297_solr x:collection1_shard1_replica_t1 s:shard1 c:collection1 r:core_node4) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.SolrCore [collection1_shard1_replica_t1] Registered new searcher Searcher@13fb019[collection1_shard1_replica_t1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2748267 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2748267 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2748272 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=2 found=1 timeoutin=9999ms
[junit4] 2> 2748287 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.s.SolrIndexSearcher Opening [Searcher@748d54[collection1_shard2_replica_t3] main]
[junit4] 2> 2748290 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
[junit4] 2> 2748291 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
[junit4] 2> 2748293 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2748294 INFO (searcherExecutor-9104-thread-1-processing-n:127.0.0.1:51297_solr x:collection1_shard2_replica_t3 s:shard2 c:collection1 r:core_node7) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.SolrCore [collection1_shard2_replica_t3] Registered new searcher Searcher@748d54[collection1_shard2_replica_t3] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2748295 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577271216579805184
[junit4] 2> 2748302 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=9999ms
[junit4] 2> 2748344 INFO (qtp14482372-23745) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2748344 INFO (qtp14482372-23745) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2748346 INFO (qtp14482372-23745) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2748347 INFO (qtp14482372-23745) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2748350 INFO (qtp14482372-23745) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.s.SolrIndexSearcher Opening [Searcher@a11915[collection1_shard2_replica_t5] main]
[junit4] 2> 2748352 INFO (qtp14482372-23745) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
[junit4] 2> 2748353 INFO (qtp14482372-23745) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
[junit4] 2> 2748355 INFO (qtp14482372-23745) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2748355 INFO (qtp14482372-23745) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577271216642719744
[junit4] 2> 2748357 INFO (searcherExecutor-9106-thread-1-processing-n:127.0.0.1:51296_solr x:collection1_shard2_replica_t5 s:shard2 c:collection1 r:core_node8) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.c.SolrCore [collection1_shard2_replica_t5] Registered new searcher Searcher@a11915[collection1_shard2_replica_t5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2748405 INFO (zkCallback-4764-thread-2-processing-n:127.0.0.1:51297_solr) [n:127.0.0.1:51297_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2748405 INFO (zkCallback-4765-thread-2-processing-n:127.0.0.1:51296_solr) [n:127.0.0.1:51296_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2748405 INFO (zkCallback-4765-thread-1-processing-n:127.0.0.1:51296_solr) [n:127.0.0.1:51296_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2748405 INFO (zkCallback-4764-thread-1-processing-n:127.0.0.1:51297_solr) [n:127.0.0.1:51297_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2748421 INFO (qtp14482372-23746) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 2748421 INFO (qtp14482372-23746) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2748424 INFO (qtp14482372-23746) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2748424 INFO (qtp14482372-23746) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2748424 INFO (qtp14482372-23746) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.s.SolrIndexSearcher Opening [Searcher@195d4c5[collection1_shard1_replica_t2] main]
[junit4] 2> 2748431 INFO (qtp14482372-23746) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
[junit4] 2> 2748432 INFO (qtp14482372-23746) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
[junit4] 2> 2748433 INFO (qtp14482372-23746) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2748434 INFO (searcherExecutor-9112-thread-1-processing-n:127.0.0.1:51296_solr x:collection1_shard1_replica_t2 s:shard1 c:collection1 r:core_node6) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.c.SolrCore [collection1_shard1_replica_t2] Registered new searcher Searcher@195d4c5[collection1_shard1_replica_t2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2748435 INFO (qtp14482372-23746) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577271216726605824
[junit4] 2> 2748774 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2748774 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2748774 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:51297/solr/collection1_shard1_replica_t1/
[junit4] 2> 2748775 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.u.PeerSync PeerSync: core=collection1_shard1_replica_t1 url=https://127.0.0.1:51297/solr START replicas=[https://127.0.0.1:51296/solr/collection1_shard1_replica_t2/] nUpdates=100
[junit4] 2> 2748786 INFO (qtp14482372-23748) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.c.S.Request [collection1_shard1_replica_t2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 2748804 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2748804 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2748804 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:51297/solr/collection1_shard2_replica_t3/
[junit4] 2> 2748805 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.u.PeerSync PeerSync: core=collection1_shard2_replica_t3 url=https://127.0.0.1:51297/solr START replicas=[https://127.0.0.1:51296/solr/collection1_shard2_replica_t5/] nUpdates=100
[junit4] 2> 2748811 INFO (qtp14482372-23740) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.c.S.Request [collection1_shard2_replica_t5] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 2749076 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.u.PeerSync PeerSync: core=collection1_shard1_replica_t1 url=https://127.0.0.1:51297/solr DONE. We have no versions. sync failed.
[junit4] 2> 2749076 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 2749076 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 2749076 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2749076 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.ZkController collection1_shard1_replica_t1 stopping background replication from leader
[junit4] 2> 2749080 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:51297/solr/collection1_shard1_replica_t1/ shard1
[junit4] 2> 2749105 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.u.PeerSync PeerSync: core=collection1_shard2_replica_t3 url=https://127.0.0.1:51297/solr DONE. We have no versions. sync failed.
[junit4] 2> 2749105 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 2749105 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 2749105 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2749106 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.ZkController collection1_shard2_replica_t3 stopping background replication from leader
[junit4] 2> 2749116 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:51297/solr/collection1_shard2_replica_t3/ shard2
[junit4] 2> 2749229 INFO (zkCallback-4764-thread-1-processing-n:127.0.0.1:51297_solr) [n:127.0.0.1:51297_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2749229 INFO (zkCallback-4765-thread-1-processing-n:127.0.0.1:51296_solr) [n:127.0.0.1:51296_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2749229 INFO (zkCallback-4765-thread-2-processing-n:127.0.0.1:51296_solr) [n:127.0.0.1:51296_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2749229 INFO (zkCallback-4764-thread-2-processing-n:127.0.0.1:51297_solr) [n:127.0.0.1:51297_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2749231 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2749235 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=config&newCollection=true&name=collection1_shard1_replica_t1&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=2543
[junit4] 2> 2749272 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2749276 INFO (qtp5470645-23739) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node7&collection.configName=config&newCollection=true&name=collection1_shard2_replica_t3&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=2583
[junit4] 2> 2749362 INFO (qtp14482372-23745) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.c.ZkController collection1_shard2_replica_t5 starting background replication from leader
[junit4] 2> 2749362 INFO (qtp14482372-23745) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
[junit4] 2> 2749369 INFO (qtp14482372-23745) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
[junit4] 2> 2749370 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher Replica core_node7 is leader but it's state is down, skipping replication
[junit4] 2> 2749372 INFO (qtp14482372-23745) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node8&collection.configName=config&newCollection=true&name=collection1_shard2_replica_t5&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=2637
[junit4] 2> 2749441 INFO (qtp14482372-23746) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.c.ZkController collection1_shard1_replica_t2 starting background replication from leader
[junit4] 2> 2749441 INFO (qtp14482372-23746) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
[junit4] 2> 2749444 INFO (qtp14482372-23746) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
[junit4] 2> 2749444 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.h.IndexFetcher Replica core_node4 is leader but it's state is down, skipping replication
[junit4] 2> 2749447 INFO (qtp14482372-23746) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node6&collection.configName=config&newCollection=true&name=collection1_shard1_replica_t2&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=2678
[junit4] 2> 2749453 INFO (qtp5470645-23737) [n:127.0.0.1:51297_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2749553 INFO (zkCallback-4764-thread-2-processing-n:127.0.0.1:51297_solr) [n:127.0.0.1:51297_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2749553 INFO (zkCallback-4765-thread-2-processing-n:127.0.0.1:51296_solr) [n:127.0.0.1:51296_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2749553 INFO (zkCallback-4765-thread-1-processing-n:127.0.0.1:51296_solr) [n:127.0.0.1:51296_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2749553 INFO (zkCallback-4764-thread-1-processing-n:127.0.0.1:51297_solr) [n:127.0.0.1:51297_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 2750346 INFO (OverseerCollectionConfigSetProcessor-98579450853654533-127.0.0.1:51297_solr-n_0000000000) [n:127.0.0.1:51297_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2750455 INFO (qtp5470645-23737) [n:127.0.0.1:51297_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&replicationFactor=0&collection.configName=config&maxShardsPerNode=2&name=collection1&nrtReplicas=0&action=CREATE&numShards=2&tlogReplicas=2&wt=javabin&version=2} status=0 QTime=4116
[junit4] 2> 2750456 INFO (SUITE-TestCloudRecovery-seed#[4EE4C0B87CE14867]-worker) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):30
[junit4] 2> 2750456 INFO (SUITE-TestCloudRecovery-seed#[4EE4C0B87CE14867]-worker) [ ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: collection1
[junit4] 2> 2750530 INFO (TEST-TestCloudRecovery.leaderRecoverFromLogOnStartupTest-seed#[4EE4C0B87CE14867]) [ ] o.a.s.SolrTestCaseJ4 ###Starting leaderRecoverFromLogOnStartupTest
[junit4] 2> 2750553 INFO (qtp14482372-23740) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_t2] webapp=/solr path=/update params={update.distrib=FROMLEADER&_version_=-1577271218933858304&distrib.from=https://127.0.0.1:51297/solr/collection1_shard1_replica_t1/&wt=javabin&version=2}{deleteByQuery=*:* (-1577271218933858304)} 0 1
[junit4] 2> 2750554 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_t1] webapp=/solr path=/update params={update.distrib=TOLEADER&distrib.from=https://127.0.0.1:51297/solr/collection1_shard2_replica_t3/&wt=javabin&version=2}{deleteByQuery=*:* (-1577271218933858304)} 0 14
[junit4] 2> 2750557 INFO (qtp14482372-23743) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_t5] webapp=/solr path=/update params={update.distrib=FROMLEADER&_version_=-1577271218926518272&distrib.from=https://127.0.0.1:51297/solr/collection1_shard2_replica_t3/&wt=javabin&version=2}{deleteByQuery=*:* (-1577271218926518272)} 0 10
[junit4] 2> 2750558 INFO (qtp5470645-23749) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_t3] webapp=/solr path=/update params={wt=javabin&version=2}{deleteByQuery=*:* (-1577271218926518272)} 0 25
[junit4] 2> 2750566 INFO (qtp14482372-23745) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.u.TestInjection Start waiting for replica in sync with leader
[junit4] 2> 2750567 INFO (qtp5470645-23737) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577271218962169856,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2750569 INFO (qtp5470645-23737) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 2750569 INFO (qtp5470645-23737) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2750570 INFO (qtp5470645-23737) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_t1] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:51297/solr/collection1_shard2_replica_t3/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 3
[junit4] 2> 2750574 INFO (qtp5470645-23737) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577271218969509888,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2750574 INFO (qtp5470645-23737) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 2750575 INFO (qtp5470645-23737) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2750576 INFO (qtp5470645-23737) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_t3] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:51297/solr/collection1_shard2_replica_t3/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 2
[junit4] 2> 2750580 INFO (qtp14482372-23746) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.u.TestInjection Start waiting for replica in sync with leader
[junit4] 2> 2750580 INFO (qtp5470645-23742) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2750588 INFO (qtp5470645-23741) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2751088 INFO (qtp5470645-23744) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2751095 INFO (qtp5470645-23737) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2751599 INFO (qtp5470645-23742) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2751606 INFO (qtp5470645-23737) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2752114 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2752124 INFO (qtp5470645-23742) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2752370 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher Last replication failed, so I'll force replication
[junit4] 2> 2752370 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher Updated masterUrl to https://127.0.0.1:51297/solr/collection1_shard2_replica_t3/
[junit4] 2> 2752373 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2752373 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 2752373 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 2752373 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 2752373 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 2752373 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher New index in Master. Deleting mine...
[junit4] 2> 2752373 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2752374 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@c66636 commitCommandVersion:0
[junit4] 2> 2752380 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.s.SolrIndexSearcher Opening [Searcher@1fb6c0d[collection1_shard2_replica_t5] main]
[junit4] 2> 2752381 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2752382 INFO (searcherExecutor-9106-thread-1-processing-n:127.0.0.1:51296_solr x:collection1_shard2_replica_t5 s:shard2 c:collection1 r:core_node8) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.c.SolrCore [collection1_shard2_replica_t5] Registered new searcher Searcher@1fb6c0d[collection1_shard2_replica_t5] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2752445 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.h.IndexFetcher Last replication failed, so I'll force replication
[junit4] 2> 2752445 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.h.IndexFetcher Updated masterUrl to https://127.0.0.1:51297/solr/collection1_shard1_replica_t1/
[junit4] 2> 2752448 INFO (qtp5470645-23737) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2752449 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 2752449 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 2752449 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 2752449 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 2752449 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.h.IndexFetcher New index in Master. Deleting mine...
[junit4] 2> 2752449 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.u.DirectUpdateHandler2 start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2752450 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@19fc5b7 commitCommandVersion:0
[junit4] 2> 2752454 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.s.SolrIndexSearcher Opening [Searcher@167aec7[collection1_shard1_replica_t2] main]
[junit4] 2> 2752455 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2752456 INFO (searcherExecutor-9112-thread-1-processing-n:127.0.0.1:51296_solr x:collection1_shard1_replica_t2 s:shard1 c:collection1 r:core_node6) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.c.SolrCore [collection1_shard1_replica_t2] Registered new searcher Searcher@167aec7[collection1_shard1_replica_t2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2752621 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2752633 INFO (qtp5470645-23742) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2753126 INFO (qtp5470645-23749) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2753139 INFO (qtp5470645-23737) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2753640 INFO (qtp5470645-23741) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2753652 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2754173 INFO (qtp5470645-23744) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2754178 INFO (qtp5470645-23741) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2754685 INFO (qtp5470645-23744) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2754691 INFO (qtp5470645-23744) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2755197 INFO (qtp5470645-23749) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2755206 INFO (qtp5470645-23741) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2755373 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2755374 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 2755374 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 2755374 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher Slave's generation: 2
[junit4] 2> 2755374 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher Slave's version: 1504203053338
[junit4] 2> 2755446 INFO (qtp5470645-23742) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2755446 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 2755446 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 2755447 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.h.IndexFetcher Slave's generation: 2
[junit4] 2> 2755447 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.h.IndexFetcher Slave's version: 1504203053414
[junit4] 2> 2755730 INFO (qtp5470645-23741) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2755730 INFO (qtp5470645-23749) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2756241 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2756243 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2756753 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2756753 INFO (qtp5470645-23749) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2757263 INFO (qtp5470645-23742) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2757263 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2757773 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2757774 INFO (qtp5470645-23742) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2758286 INFO (qtp5470645-23741) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2758304 INFO (qtp5470645-23742) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2758370 INFO (qtp5470645-23744) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2758370 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 2758371 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 2758371 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher Slave's generation: 2
[junit4] 2> 2758371 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher Slave's version: 1504203053338
[junit4] 2> 2758445 INFO (qtp5470645-23737) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2758446 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 2758446 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 2758446 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.h.IndexFetcher Slave's generation: 2
[junit4] 2> 2758446 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.h.IndexFetcher Slave's version: 1504203053414
[junit4] 2> 2758796 INFO (qtp5470645-23744) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2758812 INFO (qtp5470645-23737) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2759301 INFO (qtp5470645-23742) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2759317 INFO (qtp5470645-23741) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2759810 INFO (qtp5470645-23749) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2759827 INFO (qtp5470645-23737) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2760330 INFO (qtp5470645-23742) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2760342 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2760840 INFO (qtp5470645-23749) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2760852 INFO (qtp5470645-23737) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2761353 INFO (qtp5470645-23742) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2761364 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2761371 INFO (qtp5470645-23742) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2761371 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 2761371 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 2761372 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher Slave's generation: 2
[junit4] 2> 2761372 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher Slave's version: 1504203053338
[junit4] 2> 2761444 INFO (qtp5470645-23737) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2761445 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 2761445 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 2761445 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.h.IndexFetcher Slave's generation: 2
[junit4] 2> 2761445 INFO (indexFetcher-9127-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard1 r:core_node6 x:collection1_shard1_replica_t2] o.a.s.h.IndexFetcher Slave's version: 1504203053414
[junit4] 2> 2761864 INFO (qtp5470645-23742) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2761873 INFO (qtp5470645-23744) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2762374 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2762385 INFO (qtp5470645-23741) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2762888 INFO (qtp5470645-23737) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2762897 INFO (qtp5470645-23744) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=1
[junit4] 2> 2763396 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2763408 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2763901 INFO (qtp5470645-23749) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2763915 INFO (qtp5470645-23744) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_replica_t1] o.a.s.c.S.Request [collection1_shard1_replica_t1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2764370 INFO (qtp5470645-23749) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2764370 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher Master's generation: 1
[junit4] 2> 2764370 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher Master's version: 0
[junit4] 2> 2764370 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher Slave's generation: 2
[junit4] 2> 2764370 INFO (indexFetcher-9125-thread-1) [n:127.0.0.1:51296_solr c:collection1 s:shard2 r:core_node8 x:collection1_shard2_replica_t5] o.a.s.h.IndexFetcher Slave's version: 1504203053338
[junit4] 2> 2764408 INFO (qtp5470645-23737) [n:127.0.0.1:51297_solr c:collection1 s:shard2 r:core_node7 x:collection1_shard2_replica_t3] o.a.s.c.S.Request [collection1_shard2_replica_t3] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=details} status=0 QTime=0
[junit4] 2> 2764422 INFO (qtp5470645-23751) [n:127.0.0.1:51297_solr c:collection1 s:shard1 r:core_node4 x:collection1_shard1_re
[...truncated too long message...]
o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e397cfe490011, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 2919543 WARN (zkCallback-4790-thread-2-processing-n:127.0.0.1:51296_solr) [n:127.0.0.1:51296_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 2919543 INFO (jetty-closer-4754-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@12c3c3{/solr,null,UNAVAILABLE}
[junit4] 2> 2919544 ERROR (SUITE-TestCloudRecovery-seed#[4EE4C0B87CE14867]-worker) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2919544 INFO (SUITE-TestCloudRecovery-seed#[4EE4C0B87CE14867]-worker) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:51292 51292
[junit4] 2> 2919547 INFO (Thread-4702) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:51292 51292
[junit4] 2> 2919550 WARN (Thread-4702) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 7 /solr/aliases.json
[junit4] 2> 6 /solr/security.json
[junit4] 2> 6 /solr/configs/config
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 138 /solr/collections/collection1/state.json
[junit4] 2> 7 /solr/clusterstate.json
[junit4] 2> 7 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 7 /solr/collections
[junit4] 2> 4 /solr/overseer/queue
[junit4] 2> 4 /solr/overseer/collection-queue-work
[junit4] 2> 3 /solr/live_nodes
[junit4] 2> 2 /solr/overseer/queue-work
[junit4] 2>
[junit4] 2> NOTE: leaving temporary files on disk at: C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.TestCloudRecovery_4EE4C0B87CE14867-001
[junit4] 2> Aug 31, 2017 6:13:40 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {id=PostingsFormat(name=Direct)}, docValues:{}, maxPointsInLeafNode=1442, maxMBSortInHeap=7.733766405537039, sim=RandomSimilarity(queryNorm=true): {}, locale=no-NO, timezone=Pacific/Enderbury
[junit4] 2> NOTE: Windows 10 10.0 x86/Oracle Corporation 1.8.0_144 (32-bit)/cpus=3,threads=1,free=190939400,total=360239104
[junit4] 2> NOTE: All tests run in this JVM: [IndexBasedSpellCheckerTest, RAMDirectoryFactoryTest, TestHttpShardHandlerFactory, DefaultValueUpdateProcessorTest, DirectSolrConnectionTest, TestLockTree, TestManagedSynonymFilterFactory, TestExactSharedStatsCache, TestManagedSchema, DistribCursorPagingTest, ChaosMonkeySafeLeaderWithPullReplicasTest, TestPerFieldSimilarityWithDefaultOverride, QueryResultKeyTest, DeleteInactiveReplicaTest, HdfsRecoveryZkTest, TestDistributedSearch, TestStressInPlaceUpdates, V2StandaloneTest, TestObjectReleaseTracker, ClassificationUpdateProcessorIntegrationTest, TestFieldResource, SortSpecParsingTest, TestJettySolrRunner, XsltUpdateRequestHandlerTest, SuggesterTest, UUIDFieldTest, MoveReplicaHDFSTest, RecoveryZkTest, DistribJoinFromCollectionTest, DistributedSuggestComponentTest, TestOmitPositions, TestScoreJoinQPNoScore, ClusterStateTest, ConfigSetsAPITest, AnalyticsMergeStrategyTest, SimplePostToolTest, TestUtils, TestBinaryField, TestSchemaResource, SortByFunctionTest, TestMaxScoreQueryParser, TestAuthorizationFramework, DistributedFacetPivotLongTailTest, TestInPlaceUpdatesDistrib, TestApiFramework, PrimUtilsTest, HdfsNNFailoverTest, CleanupOldIndexTest, CursorPagingTest, AnalysisErrorHandlingTest, TestExtendedDismaxParser, TestSSLRandomization, TestDocTermOrds, BlockJoinFacetSimpleTest, TestConfigOverlay, TestStressUserVersions, CoreAdminHandlerTest, TestRandomFlRTGCloud, SolrInfoBeanTest, TestDistribDocBasedVersion, SuggesterWFSTTest, TestChildDocTransformer, TestClusterStateMutator, BlockJoinFacetDistribTest, TestSubQueryTransformerCrossCore, TestHighlightDedupGrouping, SpatialRPTFieldTypeTest, WordBreakSolrSpellCheckerTest, SubstringBytesRefFilterTest, UniqFieldsUpdateProcessorFactoryTest, HardAutoCommitTest, ResourceLoaderTest, TestQuerySenderListener, TestHdfsBackupRestoreCore, TestSolrCoreProperties, TestLRUStatsCache, TestRandomDVFaceting, CloneFieldUpdateProcessorFactoryTest, BitVectorTest, SliceStateTest, CursorMarkTest, ReturnFieldsTest, RequestLoggingTest, SolrMetricManagerTest, TestFieldCacheReopen, TestLFUCache, TestShortCircuitedRequests, HdfsWriteToMultipleCollectionsTest, TestCloudInspectUtil, TestFoldingMultitermQuery, CdcrReplicationHandlerTest, TestRawTransformer, BlockDirectoryTest, LeaderInitiatedRecoveryOnCommitTest, TestJsonFacetRefinement, AtomicUpdatesTest, ShufflingReplicaListTransformerTest, CdcrBootstrapTest, PropertiesRequestHandlerTest, TestIBSimilarityFactory, HdfsDirectoryTest, TestAuthenticationFramework, SyncSliceTest, NodeMutatorTest, TestPseudoReturnFields, QueryParsingTest, TestStressReorder, TestUniqueKeyFieldResource, TestCollationFieldDocValues, TestSystemCollAutoCreate, TestCloudPivotFacet, ConjunctionSolrSpellCheckerTest, PrimitiveFieldTypeTest, DistanceFunctionTest, JSONWriterTest, ChaosMonkeyNothingIsSafeTest, TestReloadDeadlock, TestReplicationHandler, TestReRankQParserPlugin, TestCloudNestedDocsSort, UnloadDistributedZkTest, DistributedSpellCheckComponentTest, TestWriterPerf, TestFieldTypeResource, FieldAnalysisRequestHandlerTest, TestOnReconnectListenerSupport, DateMathParserTest, TestNumericTerms32, TestMultiWordSynonyms, TestSQLHandler, TestJmxIntegration, TestLegacyNumericUtils, TestFunctionQuery, TestExportWriter, BigEndianAscendingWordDeserializerTest, TestInfoStreamLogging, TestInitQParser, SolrCoreTest, TestPhraseSuggestions, OverseerTest, TestSortByMinMaxFunction, CollectionReloadTest, HighlighterTest, OverseerModifyCollectionTest, SolrTestCaseJ4Test, TestGroupingSearch, TestDistributedStatsComponentCardinality, DistributedDebugComponentTest, TestReqParamsAPI, TestDynamicLoading, ZkFailoverTest, QueryElevationComponentTest, ScriptEngineTest, TestPayloadScoreQParserPlugin, TestCopyFieldCollectionResource, ForceLeaderTest, AlternateDirectoryTest, TestRuleBasedAuthorizationPlugin, ActionThrottleTest, MigrateRouteKeyTest, NotRequiredUniqueKeyTest, PeerSyncTest, DirectUpdateHandlerTest, ZkStateWriterTest, CircularListTest, TestHashPartitioner, SimpleMLTQParserTest, DocValuesTest, SharedFSAutoReplicaFailoverTest, PreAnalyzedUpdateProcessorTest, TestDynamicFieldResource, TestSolrCloudWithSecureImpersonation, HdfsSyncSliceTest, TestMinMaxOnMultiValuedField, TestJsonRequest, TestSolrCloudWithDelegationTokens, OverseerStatusTest, DateFieldTest, CoreSorterTest, CollectionsAPIDistributedZkTest, ClusterStateUpdateTest, LeaderElectionTest, ZkCLITest, ShardRoutingCustomTest, TermVectorComponentDistributedTest, ZkControllerTest, TestRealTimeGet, TestStressVersions, TestCoreContainer, SpellCheckComponentTest, SolrCmdDistributorTest, TestLazyCores, SignatureUpdateProcessorFactoryTest, TestCSVLoader, NoCacheHeaderTest, SolrCoreCheckLockOnStartupTest, TestUpdate, TestAtomicUpdateErrorCases, DirectUpdateHandlerOptimizeTest, TestRemoteStreaming, CacheHeaderTest, LukeRequestHandlerTest, TestQueryTypes, XmlUpdateRequestHandlerTest, RequestHandlersTest, DocumentBuilderTest, TermVectorComponentTest, MoreLikeThisHandlerTest, RegexBoostProcessorTest, IndexSchemaTest, JsonLoaderTest, TestPartialUpdateDeduplication, PingRequestHandlerTest, HighlighterConfigTest, UpdateParamsTest, TestQuerySenderNoQuery, SolrIndexConfigTest, TestMergePolicyConfig, NumericFieldsTest, TestElisionMultitermQuery, OutputWriterTest, ExternalFileFieldSortTest, TestPostingsSolrHighlighter, SpellPossibilityIteratorTest, TestCodecSupport, TestPerFieldSimilarity, OpenExchangeRatesOrgProviderTest, PluginInfoTest, TestSystemIdResolver, SpellingQueryConverterTest, ZkNodePropsTest, TestRTGBase, DistributedIntervalFacetingTest, TestTolerantSearch, AliasIntegrationTest, AssignTest, CloudExitableDirectoryReaderTest, CollectionTooManyReplicasTest, CollectionsAPISolrJTest, DeleteReplicaTest, DistributedVersionInfoTest, LeaderElectionContextKeyTest, LeaderFailoverAfterPartitionTest, MissingSegmentRecoveryTest, MoveReplicaTest, OverriddenZkACLAndCredentialsProvidersTest, OverseerTaskQueueTest, PeerSyncReplicationTest, SSLMigrationTest, SaslZkACLProviderTest, SimpleCollectionCreateDeleteTest, SolrCLIZkUtilsTest, SolrCloudExampleTest, TestCloudDeleteByQuery, TestCloudRecovery]
[junit4] Completed [535/732 (1!)] on J0 in 174.60s, 2 tests, 1 failure <<< FAILURES!
[...truncated 42652 lines...]
BUILD FAILED
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\build.xml:810: The following error occurred while executing this line:
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\build.xml:690: The following error occurred while executing this line:
C:\Users\jenkins\workspace\Lucene-Solr-master-Windows\build.xml:678: Source checkout is modified!!! Offending files:
* solr/licenses/httpmime-4.5.3.jar.sha1
Total time: 88 minutes 52 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
[WARNINGS] Skipping publisher since build result is FAILURE
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any