You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2017/08/30 23:32:48 UTC
[JENKINS] Lucene-Solr-7.0-Windows (64bit/jdk1.8.0_144) - Build #
122 - Still Unstable!
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.0-Windows/122/
Java: 64bit/jdk1.8.0_144 -XX:-UseCompressedOops -XX:+UseParallelGC
1 tests failed.
FAILED: junit.framework.TestSuite.org.apache.solr.cloud.TestCloudRecovery
Error Message:
Could not remove the following files (in the order of attempts): C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1\data\tlog\tlog.0000000000000000001: java.nio.file.FileSystemException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1\data\tlog\tlog.0000000000000000001: The process cannot access the file because it is being used by another process. C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1\data\tlog: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1\data\tlog C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1\data: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1\data C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1 C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1\data\tlog\tlog.0000000000000000001: java.nio.file.FileSystemException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1\data\tlog\tlog.0000000000000000001: The process cannot access the file because it is being used by another process. C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1\data\tlog: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1\data\tlog C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1\data: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1\data C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1 C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1 C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2\data\tlog\tlog.0000000000000000001: java.nio.file.FileSystemException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2\data\tlog\tlog.0000000000000000001: The process cannot access the file because it is being used by another process. C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2\data\tlog: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2\data\tlog C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2\data: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2\data C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2 C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2\data\tlog\tlog.0000000000000000001: java.nio.file.FileSystemException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2\data\tlog\tlog.0000000000000000001: The process cannot access the file because it is being used by another process. C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2\data\tlog: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2\data\tlog C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2\data: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2\data C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2 C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2 C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001 C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001
Stack Trace:
java.io.IOException: Could not remove the following files (in the order of attempts):
C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1\data\tlog\tlog.0000000000000000001: java.nio.file.FileSystemException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1\data\tlog\tlog.0000000000000000001: The process cannot access the file because it is being used by another process.
C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1\data\tlog: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1\data\tlog
C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1\data: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1\data
C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1
C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1\data\tlog\tlog.0000000000000000001: java.nio.file.FileSystemException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1\data\tlog\tlog.0000000000000000001: The process cannot access the file because it is being used by another process.
C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1\data\tlog: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1\data\tlog
C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1\data: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1\data
C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1
C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1
C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2\data\tlog\tlog.0000000000000000001: java.nio.file.FileSystemException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2\data\tlog\tlog.0000000000000000001: The process cannot access the file because it is being used by another process.
C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2\data\tlog: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2\data\tlog
C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2\data: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2\data
C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2
C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2\data\tlog\tlog.0000000000000000001: java.nio.file.FileSystemException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2\data\tlog\tlog.0000000000000000001: The process cannot access the file because it is being used by another process.
C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2\data\tlog: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2\data\tlog
C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2\data: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2\data
C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2
C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2
C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001
C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001
at __randomizedtesting.SeedInfo.seed([E123E0CB727F2B47]:0)
at org.apache.lucene.util.IOUtils.rm(IOUtils.java:329)
at org.apache.lucene.util.TestRuleTemporaryFilesCleanup.afterAlways(TestRuleTemporaryFilesCleanup.java:216)
at com.carrotsearch.randomizedtesting.rules.TestRuleAdapter$1.afterAlways(TestRuleAdapter.java:31)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:43)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 11162 lines...]
[junit4] Suite: org.apache.solr.cloud.TestCloudRecovery
[junit4] 2> Creating dataDir: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\init-core-data-001
[junit4] 2> 279397 WARN (SUITE-TestCloudRecovery-seed#[E123E0CB727F2B47]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=1 numCloses=1
[junit4] 2> 279397 INFO (SUITE-TestCloudRecovery-seed#[E123E0CB727F2B47]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 279400 INFO (SUITE-TestCloudRecovery-seed#[E123E0CB727F2B47]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 279402 INFO (SUITE-TestCloudRecovery-seed#[E123E0CB727F2B47]-worker) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001
[junit4] 2> 279402 INFO (SUITE-TestCloudRecovery-seed#[E123E0CB727F2B47]-worker) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 279402 INFO (Thread-788) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 279402 INFO (Thread-788) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 279405 ERROR (Thread-788) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 279503 INFO (SUITE-TestCloudRecovery-seed#[E123E0CB727F2B47]-worker) [ ] o.a.s.c.ZkTestServer start zk server on port:63719
[junit4] 2> 279517 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e3542da9c0000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 279522 INFO (jetty-launcher-431-thread-2) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 279522 INFO (jetty-launcher-431-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 279525 INFO (jetty-launcher-431-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@44d8a6e4{/solr,null,AVAILABLE}
[junit4] 2> 279526 INFO (jetty-launcher-431-thread-2) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@35140d73{/solr,null,AVAILABLE}
[junit4] 2> 279527 INFO (jetty-launcher-431-thread-2) [ ] o.e.j.s.AbstractConnector Started ServerConnector@130327de{HTTP/1.1,[http/1.1]}{127.0.0.1:63724}
[junit4] 2> 279527 INFO (jetty-launcher-431-thread-2) [ ] o.e.j.s.Server Started @283137ms
[junit4] 2> 279527 INFO (jetty-launcher-431-thread-2) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=63724}
[junit4] 2> 279527 INFO (jetty-launcher-431-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@60bdf76{HTTP/1.1,[http/1.1]}{127.0.0.1:63723}
[junit4] 2> 279533 INFO (jetty-launcher-431-thread-1) [ ] o.e.j.s.Server Started @283143ms
[junit4] 2> 279533 INFO (jetty-launcher-431-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=63723}
[junit4] 2> 279533 ERROR (jetty-launcher-431-thread-2) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 279533 INFO (jetty-launcher-431-thread-2) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 279533 INFO (jetty-launcher-431-thread-2) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 279533 ERROR (jetty-launcher-431-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 279533 INFO (jetty-launcher-431-thread-2) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 279533 INFO (jetty-launcher-431-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 279534 INFO (jetty-launcher-431-thread-2) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-30T22:28:47.516Z
[junit4] 2> 279534 INFO (jetty-launcher-431-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 279534 INFO (jetty-launcher-431-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 279534 INFO (jetty-launcher-431-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-30T22:28:47.517Z
[junit4] 2> 279543 INFO (jetty-launcher-431-thread-2) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 279543 INFO (jetty-launcher-431-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 279563 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e3542da9c0001, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 279576 INFO (jetty-launcher-431-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:63719/solr
[junit4] 2> 279576 INFO (jetty-launcher-431-thread-2) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:63719/solr
[junit4] 2> 279584 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e3542da9c0003, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 279585 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e3542da9c0004, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 279709 INFO (jetty-launcher-431-thread-1) [n:127.0.0.1:63723_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 279709 INFO (jetty-launcher-431-thread-2) [n:127.0.0.1:63724_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 279710 INFO (jetty-launcher-431-thread-1) [n:127.0.0.1:63723_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:63723_solr
[junit4] 2> 279710 INFO (jetty-launcher-431-thread-2) [n:127.0.0.1:63724_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63724_solr
[junit4] 2> 279711 INFO (jetty-launcher-431-thread-1) [n:127.0.0.1:63723_solr ] o.a.s.c.Overseer Overseer (id=98574803100499974-127.0.0.1:63723_solr-n_0000000000) starting
[junit4] 2> 279713 INFO (zkCallback-442-thread-1-processing-n:127.0.0.1:63724_solr) [n:127.0.0.1:63724_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 279713 INFO (zkCallback-443-thread-1-processing-n:127.0.0.1:63723_solr) [n:127.0.0.1:63723_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 279717 INFO (jetty-launcher-431-thread-1) [n:127.0.0.1:63723_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:63723_solr
[junit4] 2> 279769 INFO (zkCallback-442-thread-1-processing-n:127.0.0.1:63724_solr) [n:127.0.0.1:63724_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 279769 INFO (zkCallback-443-thread-1-processing-n:127.0.0.1:63723_solr) [n:127.0.0.1:63723_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
[junit4] 2> 279849 INFO (jetty-launcher-431-thread-2) [n:127.0.0.1:63724_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63724.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a59f1c9
[junit4] 2> 279852 INFO (jetty-launcher-431-thread-1) [n:127.0.0.1:63723_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63723.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a59f1c9
[junit4] 2> 280222 INFO (jetty-launcher-431-thread-2) [n:127.0.0.1:63724_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63724.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a59f1c9
[junit4] 2> 280223 INFO (jetty-launcher-431-thread-2) [n:127.0.0.1:63724_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63724.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a59f1c9
[junit4] 2> 280225 INFO (jetty-launcher-431-thread-1) [n:127.0.0.1:63723_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63723.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a59f1c9
[junit4] 2> 280225 INFO (jetty-launcher-431-thread-1) [n:127.0.0.1:63723_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63723.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a59f1c9
[junit4] 2> 280228 INFO (jetty-launcher-431-thread-1) [n:127.0.0.1:63723_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\.
[junit4] 2> 280228 INFO (jetty-launcher-431-thread-2) [n:127.0.0.1:63724_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\.
[junit4] 2> 280266 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e3542da9c0007, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 280274 INFO (SUITE-TestCloudRecovery-seed#[E123E0CB727F2B47]-worker) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
[junit4] 2> 280274 INFO (SUITE-TestCloudRecovery-seed#[E123E0CB727F2B47]-worker) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:63719/solr ready
[junit4] 2> 280287 INFO (qtp482505108-2843) [n:127.0.0.1:63723_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&replicationFactor=2&collection.configName=config&maxShardsPerNode=2&name=collection1&nrtReplicas=2&action=CREATE&numShards=2&tlogReplicas=0&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 280290 INFO (OverseerThreadFactory-1362-thread-1-processing-n:127.0.0.1:63723_solr) [n:127.0.0.1:63723_solr ] o.a.s.c.CreateCollectionCmd Create collection collection1
[junit4] 2> 280396 INFO (OverseerStateUpdate-98574803100499974-127.0.0.1:63723_solr-n_0000000000) [n:127.0.0.1:63723_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:63723/solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 280400 INFO (OverseerStateUpdate-98574803100499974-127.0.0.1:63723_solr-n_0000000000) [n:127.0.0.1:63723_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"collection1_shard1_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:63724/solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 280401 INFO (OverseerStateUpdate-98574803100499974-127.0.0.1:63723_solr-n_0000000000) [n:127.0.0.1:63723_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"collection1_shard2_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:63723/solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 280402 INFO (OverseerStateUpdate-98574803100499974-127.0.0.1:63723_solr-n_0000000000) [n:127.0.0.1:63723_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"collection1",
[junit4] 2> "shard":"shard2",
[junit4] 2> "core":"collection1_shard2_replica_n2",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:63724/solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 280605 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node1&collection.configName=config&newCollection=true&name=collection1_shard1_replica_n1&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 280605 INFO (qtp1151186768-2829) [n:127.0.0.1:63724_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=config&newCollection=true&name=collection1_shard2_replica_n2&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 280606 INFO (qtp1151186768-2832) [n:127.0.0.1:63724_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=config&newCollection=true&name=collection1_shard1_replica_n2&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 280607 INFO (qtp1151186768-2829) [n:127.0.0.1:63724_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 280607 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 280607 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=config&newCollection=true&name=collection1_shard2_replica_n1&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 280728 INFO (zkCallback-442-thread-1-processing-n:127.0.0.1:63724_solr) [n:127.0.0.1:63724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 280728 INFO (zkCallback-443-thread-2-processing-n:127.0.0.1:63723_solr) [n:127.0.0.1:63723_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 280728 INFO (zkCallback-443-thread-1-processing-n:127.0.0.1:63723_solr) [n:127.0.0.1:63723_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 280728 INFO (zkCallback-442-thread-2-processing-n:127.0.0.1:63724_solr) [n:127.0.0.1:63724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 281641 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 281641 INFO (qtp1151186768-2829) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 281641 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 281654 INFO (qtp1151186768-2832) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 281656 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.s.IndexSchema [collection1_shard2_replica_n1] Schema name=minimal
[junit4] 2> 281656 INFO (qtp1151186768-2829) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.IndexSchema [collection1_shard2_replica_n2] Schema name=minimal
[junit4] 2> 281659 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema [collection1_shard1_replica_n1] Schema name=minimal
[junit4] 2> 281661 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 281661 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n1' using configuration from collection collection1, trusted=true
[junit4] 2> 281661 INFO (qtp1151186768-2832) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.s.IndexSchema [collection1_shard1_replica_n2] Schema name=minimal
[junit4] 2> 281661 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63723.solr.core.collection1.shard2.replica_n1' (registry 'solr.core.collection1.shard2.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a59f1c9
[junit4] 2> 281661 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 281662 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n1' using configuration from collection collection1, trusted=true
[junit4] 2> 281662 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63723.solr.core.collection1.shard1.replica_n1' (registry 'solr.core.collection1.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a59f1c9
[junit4] 2> 281664 INFO (qtp1151186768-2832) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 281664 INFO (qtp1151186768-2829) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 281664 INFO (qtp1151186768-2829) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard2_replica_n2' using configuration from collection collection1, trusted=true
[junit4] 2> 281664 INFO (qtp1151186768-2832) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'collection1_shard1_replica_n2' using configuration from collection collection1, trusted=true
[junit4] 2> 281664 INFO (qtp1151186768-2829) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63724.solr.core.collection1.shard2.replica_n2' (registry 'solr.core.collection1.shard2.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a59f1c9
[junit4] 2> 281664 INFO (qtp1151186768-2832) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_63724.solr.core.collection1.shard1.replica_n2' (registry 'solr.core.collection1.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@2a59f1c9
[junit4] 2> 281684 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 281684 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.c.SolrCore [[collection1_shard2_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\.\collection1_shard2_replica_n1\data\]
[junit4] 2> 281685 INFO (qtp1151186768-2829) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 281685 INFO (qtp1151186768-2829) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrCore [[collection1_shard2_replica_n2] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\.\collection1_shard2_replica_n2\data\]
[junit4] 2> 281685 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 281685 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore [[collection1_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\.\collection1_shard1_replica_n1\data\]
[junit4] 2> 281687 INFO (qtp1151186768-2832) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 281687 INFO (qtp1151186768-2832) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.c.SolrCore [[collection1_shard1_replica_n2] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\.\collection1_shard1_replica_n2\data\]
[junit4] 2> 281799 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 281799 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 281800 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 281800 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 281800 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 281800 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 281802 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 281802 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 281805 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@7e071bce[collection1_shard2_replica_n1] main]
[junit4] 2> 281805 INFO (qtp1151186768-2829) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 281805 INFO (qtp1151186768-2829) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 281806 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
[junit4] 2> 281806 INFO (qtp1151186768-2829) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 281806 INFO (qtp1151186768-2829) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 281806 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
[junit4] 2> 281807 INFO (qtp1151186768-2832) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 281807 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 281807 INFO (qtp1151186768-2832) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 281807 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577196852126679040
[junit4] 2> 281808 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@404a0cc4[collection1_shard1_replica_n1] main]
[junit4] 2> 281808 INFO (qtp1151186768-2832) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 281808 INFO (qtp1151186768-2832) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 281810 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
[junit4] 2> 281810 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
[junit4] 2> 281811 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 281812 INFO (qtp1151186768-2829) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@7bbf1582[collection1_shard2_replica_n2] main]
[junit4] 2> 281812 INFO (searcherExecutor-1367-thread-1-processing-n:127.0.0.1:63723_solr x:collection1_shard2_replica_n1 s:shard2 c:collection1 r:core_node3) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.c.SolrCore [collection1_shard2_replica_n1] Registered new searcher Searcher@7e071bce[collection1_shard2_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 281812 INFO (searcherExecutor-1368-thread-1-processing-n:127.0.0.1:63723_solr x:collection1_shard1_replica_n1 s:shard1 c:collection1 r:core_node1) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.c.SolrCore [collection1_shard1_replica_n1] Registered new searcher Searcher@404a0cc4[collection1_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 281812 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577196852131921920
[junit4] 2> 281813 INFO (qtp1151186768-2829) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
[junit4] 2> 281813 INFO (qtp1151186768-2829) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
[junit4] 2> 281814 INFO (qtp1151186768-2832) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@139cd627[collection1_shard1_replica_n2] main]
[junit4] 2> 281814 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard2: total=2 found=1 timeoutin=9999ms
[junit4] 2> 281815 INFO (qtp1151186768-2829) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 281815 INFO (qtp1151186768-2832) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/config
[junit4] 2> 281815 INFO (searcherExecutor-1370-thread-1-processing-n:127.0.0.1:63724_solr x:collection1_shard2_replica_n2 s:shard2 c:collection1 r:core_node4) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrCore [collection1_shard2_replica_n2] Registered new searcher Searcher@7bbf1582[collection1_shard2_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 281815 INFO (qtp1151186768-2829) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577196852135067648
[junit4] 2> 281816 INFO (qtp1151186768-2832) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/config
[junit4] 2> 281816 INFO (qtp1151186768-2832) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 281816 INFO (qtp1151186768-2832) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577196852136116224
[junit4] 2> 281817 INFO (searcherExecutor-1369-thread-1-processing-n:127.0.0.1:63724_solr x:collection1_shard1_replica_n2 s:shard1 c:collection1 r:core_node2) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.c.SolrCore [collection1_shard1_replica_n2] Registered new searcher Searcher@139cd627[collection1_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 281819 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 281819 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 281819 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:63723/solr/collection1_shard1_replica_n1/
[junit4] 2> 281820 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=collection1_shard1_replica_n1 url=http://127.0.0.1:63723/solr START replicas=[http://127.0.0.1:63724/solr/collection1_shard1_replica_n2/] nUpdates=100
[junit4] 2> 281820 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.u.PeerSync PeerSync: core=collection1_shard1_replica_n1 url=http://127.0.0.1:63723/solr DONE. We have no versions. sync failed.
[junit4] 2> 281822 INFO (qtp1151186768-2840) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.c.S.Request [collection1_shard1_replica_n2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 281823 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 281823 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 281823 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 281827 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:63723/solr/collection1_shard1_replica_n1/ shard1
[junit4] 2> 281930 INFO (zkCallback-443-thread-1-processing-n:127.0.0.1:63723_solr) [n:127.0.0.1:63723_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 281930 INFO (zkCallback-443-thread-2-processing-n:127.0.0.1:63723_solr) [n:127.0.0.1:63723_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 281930 INFO (zkCallback-442-thread-2-processing-n:127.0.0.1:63724_solr) [n:127.0.0.1:63724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 281931 INFO (zkCallback-442-thread-1-processing-n:127.0.0.1:63724_solr) [n:127.0.0.1:63724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 281979 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 281982 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node1&collection.configName=config&newCollection=true&name=collection1_shard1_replica_n1&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1377
[junit4] 2> 282088 INFO (zkCallback-443-thread-2-processing-n:127.0.0.1:63723_solr) [n:127.0.0.1:63723_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 282089 INFO (zkCallback-442-thread-2-processing-n:127.0.0.1:63724_solr) [n:127.0.0.1:63724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 282088 INFO (zkCallback-442-thread-1-processing-n:127.0.0.1:63724_solr) [n:127.0.0.1:63724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 282089 INFO (zkCallback-443-thread-1-processing-n:127.0.0.1:63723_solr) [n:127.0.0.1:63723_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 282315 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 282316 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 282316 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:63723/solr/collection1_shard2_replica_n1/
[junit4] 2> 282316 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.PeerSync PeerSync: core=collection1_shard2_replica_n1 url=http://127.0.0.1:63723/solr START replicas=[http://127.0.0.1:63724/solr/collection1_shard2_replica_n2/] nUpdates=100
[junit4] 2> 282316 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.PeerSync PeerSync: core=collection1_shard2_replica_n1 url=http://127.0.0.1:63723/solr DONE. We have no versions. sync failed.
[junit4] 2> 282318 INFO (qtp1151186768-2838) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.S.Request [collection1_shard2_replica_n2] webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 282319 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
[junit4] 2> 282319 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
[junit4] 2> 282319 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 282325 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:63723/solr/collection1_shard2_replica_n1/ shard2
[junit4] 2> 282430 INFO (zkCallback-443-thread-1-processing-n:127.0.0.1:63723_solr) [n:127.0.0.1:63723_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 282431 INFO (zkCallback-442-thread-1-processing-n:127.0.0.1:63724_solr) [n:127.0.0.1:63724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 282431 INFO (zkCallback-443-thread-2-processing-n:127.0.0.1:63723_solr) [n:127.0.0.1:63723_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 282431 INFO (zkCallback-442-thread-2-processing-n:127.0.0.1:63724_solr) [n:127.0.0.1:63724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 282476 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 282478 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=config&newCollection=true&name=collection1_shard2_replica_n1&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1869
[junit4] 2> 282581 INFO (zkCallback-443-thread-1-processing-n:127.0.0.1:63723_solr) [n:127.0.0.1:63723_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 282581 INFO (zkCallback-443-thread-2-processing-n:127.0.0.1:63723_solr) [n:127.0.0.1:63723_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 282581 INFO (zkCallback-442-thread-2-processing-n:127.0.0.1:63724_solr) [n:127.0.0.1:63724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 282581 INFO (zkCallback-442-thread-1-processing-n:127.0.0.1:63724_solr) [n:127.0.0.1:63724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 282822 INFO (qtp1151186768-2832) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node2&collection.configName=config&newCollection=true&name=collection1_shard1_replica_n2&action=CREATE&numShards=2&collection=collection1&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2215
[junit4] 2> 282823 INFO (qtp1151186768-2829) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=config&newCollection=true&name=collection1_shard2_replica_n2&action=CREATE&numShards=2&collection=collection1&shard=shard2&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2217
[junit4] 2> 282826 INFO (qtp482505108-2843) [n:127.0.0.1:63723_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 282927 INFO (zkCallback-443-thread-1-processing-n:127.0.0.1:63723_solr) [n:127.0.0.1:63723_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 282927 INFO (zkCallback-442-thread-1-processing-n:127.0.0.1:63724_solr) [n:127.0.0.1:63724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 282927 INFO (zkCallback-443-thread-2-processing-n:127.0.0.1:63723_solr) [n:127.0.0.1:63723_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 282927 INFO (zkCallback-442-thread-2-processing-n:127.0.0.1:63724_solr) [n:127.0.0.1:63724_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/collection1/state.json] for collection [collection1] has occurred - updating... (live nodes size: [2])
[junit4] 2> 283826 INFO (qtp482505108-2843) [n:127.0.0.1:63723_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&replicationFactor=2&collection.configName=config&maxShardsPerNode=2&name=collection1&nrtReplicas=2&action=CREATE&numShards=2&tlogReplicas=0&wt=javabin&version=2} status=0 QTime=3539
[junit4] 2> 283826 INFO (SUITE-TestCloudRecovery-seed#[E123E0CB727F2B47]-worker) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: collection1 failOnTimeout:true timeout (sec):30
[junit4] 2> 283827 INFO (SUITE-TestCloudRecovery-seed#[E123E0CB727F2B47]-worker) [ ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: collection1
[junit4] 2> 283846 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[E123E0CB727F2B47]) [ ] o.a.s.SolrTestCaseJ4 ###Starting corruptedLogTest
[junit4] 2> 283869 INFO (qtp1151186768-2830) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&_version_=-1577196854269968384&distrib.from=http://127.0.0.1:63723/solr/collection1_shard2_replica_n1/&wt=javabin&version=2}{deleteByQuery=*:* (-1577196854269968384)} 0 9
[junit4] 2> 283869 INFO (qtp482505108-2839) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n1] webapp=/solr path=/update params={update.distrib=TOLEADER&distrib.from=http://127.0.0.1:63723/solr/collection1_shard1_replica_n1/&wt=javabin&version=2}{deleteByQuery=*:* (-1577196854269968384)} 0 18
[junit4] 2> 283870 INFO (qtp1151186768-2842) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&_version_=-1577196854265774080&distrib.from=http://127.0.0.1:63723/solr/collection1_shard1_replica_n1/&wt=javabin&version=2}{deleteByQuery=*:* (-1577196854265774080)} 0 13
[junit4] 2> 283870 INFO (qtp482505108-2837) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{deleteByQuery=*:* (-1577196854265774080)} 0 23
[junit4] 2> 283873 INFO (qtp1151186768-2833) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577196854293037056,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 283873 INFO (qtp1151186768-2833) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 283876 INFO (qtp1151186768-2833) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 283876 INFO (qtp1151186768-2840) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577196854296182784,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 283876 INFO (qtp1151186768-2840) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 283876 INFO (qtp1151186768-2833) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:63723/solr/collection1_shard2_replica_n1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 3
[junit4] 2> 283876 INFO (qtp1151186768-2840) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 283878 INFO (qtp482505108-2835) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577196854298279936,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 283878 INFO (qtp482505108-2835) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 283880 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577196854300377088,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 283880 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 283880 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 283881 INFO (qtp482505108-2835) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 283881 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:63723/solr/collection1_shard2_replica_n1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 0
[junit4] 2> 283885 INFO (qtp1151186768-2840) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:63723/solr/collection1_shard2_replica_n1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 5
[junit4] 2> 283885 INFO (qtp482505108-2835) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n1] webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=http://127.0.0.1:63723/solr/collection1_shard2_replica_n1/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 8
[junit4] 2> 283885 INFO (qtp482505108-2836) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n1] webapp=/solr path=/update params={_stateVer_=collection1:7&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 14
[junit4] 2> 283895 INFO (qtp1151186768-2833) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:63723/solr/collection1_shard1_replica_n1/&wt=javabin&version=2}{add=[1000 (1577196854308765696)]} 0 3
[junit4] 2> 283896 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1000 (1577196854308765696)]} 0 8
[junit4] 2> 283904 INFO (qtp1151186768-2832) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:63723/solr/collection1_shard2_replica_n1/&wt=javabin&version=2}{add=[1001 (1577196854317154304)]} 0 3
[junit4] 2> 283904 INFO (qtp482505108-2843) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1001 (1577196854317154304)]} 0 7
[junit4] 2> 283906 INFO (qtp1151186768-2829) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:63723/solr/collection1_shard1_replica_n1/&wt=javabin&version=2}{add=[0 (1577196854326591488)]} 0 0
[junit4] 2> 283906 INFO (qtp482505108-2839) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[0 (1577196854326591488)]} 0 1
[junit4] 2> 283908 INFO (qtp1151186768-2829) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:63723/solr/collection1_shard1_replica_n1/&wt=javabin&version=2}{add=[1 (1577196854328688640)]} 0 0
[junit4] 2> 283908 INFO (qtp482505108-2839) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[1 (1577196854328688640)]} 0 1
[junit4] 2> 283909 INFO (qtp1151186768-2829) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:63723/solr/collection1_shard2_replica_n1/&wt=javabin&version=2}{add=[2 (1577196854329737216)]} 0 0
[junit4] 2> 283910 INFO (qtp482505108-2839) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[2 (1577196854329737216)]} 0 1
[junit4] 2> 283914 INFO (qtp1151186768-2829) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:63723/solr/collection1_shard2_replica_n1/&wt=javabin&version=2}{add=[3 (1577196854331834368)]} 0 2
[junit4] 2> 283915 INFO (qtp482505108-2841) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[3 (1577196854331834368)]} 0 4
[junit4] 2> 283918 INFO (qtp1151186768-2838) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:63723/solr/collection1_shard1_replica_n1/&wt=javabin&version=2}{add=[4 (1577196854338125824)]} 0 0
[junit4] 2> 283918 INFO (qtp482505108-2836) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[4 (1577196854338125824)]} 0 2
[junit4] 2> 283920 INFO (qtp1151186768-2838) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:63723/solr/collection1_shard2_replica_n1/&wt=javabin&version=2}{add=[5 (1577196854341271552)]} 0 0
[junit4] 2> 283920 INFO (qtp482505108-2836) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[5 (1577196854341271552)]} 0 1
[junit4] 2> 283922 INFO (qtp1151186768-2838) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:63723/solr/collection1_shard2_replica_n1/&wt=javabin&version=2}{add=[6 (1577196854343368704)]} 0 0
[junit4] 2> 283922 INFO (qtp482505108-2836) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[6 (1577196854343368704)]} 0 1
[junit4] 2> 283923 INFO (qtp1151186768-2838) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:63723/solr/collection1_shard2_replica_n1/&wt=javabin&version=2}{add=[7 (1577196854345465856)]} 0 0
[junit4] 2> 283923 INFO (qtp482505108-2836) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[7 (1577196854345465856)]} 0 1
[junit4] 2> 283925 INFO (qtp1151186768-2838) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:63723/solr/collection1_shard1_replica_n1/&wt=javabin&version=2}{add=[8 (1577196854346514432)]} 0 0
[junit4] 2> 283925 INFO (qtp482505108-2836) [n:127.0.0.1:63723_solr c:collection1 s:shard1 r:core_node1 x:collection1_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard1_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[8 (1577196854346514432)]} 0 1
[junit4] 2> 283926 INFO (qtp1151186768-2838) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n2] webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=http://127.0.0.1:63723/solr/collection1_shard2_replica_n1/&wt=javabin&version=2}{add=[9 (1577196854347563008)]} 0 0
[junit4] 2> 283926 INFO (qtp482505108-2836) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [collection1_shard2_replica_n1] webapp=/solr path=/update params={wt=javabin&version=2}{add=[9 (1577196854347563008)]} 0 1
[junit4] 2> 283928 INFO (qtp1151186768-2838) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.S.Request [collection1_shard2_replica_n2] webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=collection1:7&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=http://127.0.0.1:63724/solr/collection1_shard2_replica_n2/|http://127.0.0.1:63723/solr/collection1_shard2_replica_n1/&rows=10&version=2&q=*:*&NOW=1504132131910&isShard=true&wt=javabin} hits=0 status=0 QTime=0
[junit4] 2> 283929 INFO (qtp1151186768-2832) [n:127.0.0.1:63724_solr c:collection1 s:shard1 r:core_node2 x:collection1_shard1_replica_n2] o.a.s.c.S.Request [collection1_shard1_replica_n2] webapp=/solr path=/select params={df=text&distrib=false&_stateVer_=collection1:7&fl=id&fl=score&shards.purpose=4&start=0&fsv=true&shard.url=http://127.0.0.1:63724/solr/collection1_shard1_replica_n2/|http://127.0.0.1:63723/solr/collection1_shard1_replica_n1/&rows=10&version=2&q=*:*&NOW=1504132131910&isShard=true&wt=javabin} hits=0 status=0 QTime=0
[junit4] 2> 283929 INFO (qtp482505108-2834) [n:127.0.0.1:63723_solr c:collection1 s:shard2 r:core_node3 x:collection1_shard2_replica_n1] o.a.s.c.S.Request [collection1_shard2_replica_n1] webapp=/solr path=/select params={q=*:*&_stateVer_=collection1:7&wt=javabin&version=2} hits=0 status=0 QTime=2
[junit4] 2> 283931 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[E123E0CB727F2B47]) [ ] o.a.s.c.ChaosMonkey monkey: stop jetty! 63724
[junit4] 2> 283931 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[E123E0CB727F2B47]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=2002337638
[junit4] 2> 283931 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[E123E0CB727F2B47]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 283931 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[E123E0CB727F2B47]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@52c13f2: rootName = solr_63724, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@43ab63d6
[junit4] 2> 283935 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[E123E0CB727F2B47]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 283935 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[E123E0CB727F2B47]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@54a42b71: rootName = solr_63724, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@21ef4176
[junit4] 2> 283938 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[E123E0CB727F2B47]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 283938 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[E123E0CB727F2B47]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@15466cb3: rootName = solr_63724, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@68d1801a
[junit4] 2> 283939 INFO (TEST-TestCloudRecovery.corruptedLogTest-seed#[E123E0CB727F2B47]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 283944 INFO (coreCloseExecutor-1387-thread-1) [n:127.0.0.1:63724_solr c:collection1 s:shard2 r:core_node4 x:collection1_shard2_replica_n2] o.a.s.c.SolrCore [co
[...truncated too long message...]
ter, FullHLLTest, TestConfigSetImmutable, TestConfig, TestRTGBase, TestRandomFlRTGCloud, CdcrReplicationDistributedZkTest, TestInitParams, TestFastOutputStream, TestAuthorizationFramework, TestReloadDeadlock, TestXmlQParser, SortSpecParsingTest, TestCloudRecovery]
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestCloudRecovery -Dtests.seed=E123E0CB727F2B47 -Dtests.slow=true -Dtests.locale=zh-HK -Dtests.timezone=CNT -Dtests.asserts=true -Dtests.file.encoding=UTF-8
[junit4] ERROR 0.00s J1 | TestCloudRecovery (suite) <<<
[junit4] > Throwable #1: java.io.IOException: Could not remove the following files (in the order of attempts):
[junit4] > C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1\data\tlog\tlog.0000000000000000001: java.nio.file.FileSystemException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1\data\tlog\tlog.0000000000000000001: The process cannot access the file because it is being used by another process.
[junit4] > C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1\data\tlog: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1\data\tlog
[junit4] > C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1\data: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1\data
[junit4] > C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard2_replica_n1
[junit4] > C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1\data\tlog\tlog.0000000000000000001: java.nio.file.FileSystemException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1\data\tlog\tlog.0000000000000000001: The process cannot access the file because it is being used by another process.
[junit4] > C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1\data\tlog: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1\data\tlog
[junit4] > C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1\data: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1\data
[junit4] > C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1\collection1_shard1_replica_n1
[junit4] > C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node1
[junit4] > C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2\data\tlog\tlog.0000000000000000001: java.nio.file.FileSystemException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2\data\tlog\tlog.0000000000000000001: The process cannot access the file because it is being used by another process.
[junit4] > C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2\data\tlog: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2\data\tlog
[junit4] > C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2\data: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2\data
[junit4] > C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard2_replica_n2
[junit4] > C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2\data\tlog\tlog.0000000000000000001: java.nio.file.FileSystemException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2\data\tlog\tlog.0000000000000000001: The process cannot access the file because it is being used by another process.
[junit4] > C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2\data\tlog: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2\data\tlog
[junit4] > C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2\data: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2\data
[junit4] > C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2\collection1_shard1_replica_n2
[junit4] > C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001\node2
[junit4] > C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001\tempDir-001
[junit4] > C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001: java.nio.file.DirectoryNotEmptyException: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.TestCloudRecovery_E123E0CB727F2B47-001
[junit4] > at __randomizedtesting.SeedInfo.seed([E123E0CB727F2B47]:0)
[junit4] > at org.apache.lucene.util.IOUtils.rm(IOUtils.java:329)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] Completed [105/729 (1!)] on J1 in 21.76s, 2 tests, 1 error <<< FAILURES!
[...truncated 45489 lines...]
[JENKINS] Lucene-Solr-7.0-Windows (32bit/jdk1.8.0_144) - Build #
125 - Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.0-Windows/125/
Java: 32bit/jdk1.8.0_144 -client -XX:+UseParallelGC
1 tests failed.
FAILED: org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster
Error Message:
Document mismatch on target after sync expected:<2000> but was:<1100>
Stack Trace:
java.lang.AssertionError: Document mismatch on target after sync expected:<2000> but was:<1100>
at __randomizedtesting.SeedInfo.seed([82331C9EEDA821A4:567657C70AFE925F]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster(CdcrBootstrapTest.java:309)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12537 lines...]
[junit4] Suite: org.apache.solr.cloud.CdcrBootstrapTest
[junit4] 2> Creating dataDir: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrBootstrapTest_82331C9EEDA821A4-001\init-core-data-001
[junit4] 2> 2235083 WARN (SUITE-CdcrBootstrapTest-seed#[82331C9EEDA821A4]-worker) [ ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=2 numCloses=2
[junit4] 2> 2235083 INFO (SUITE-CdcrBootstrapTest-seed#[82331C9EEDA821A4]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 2235086 INFO (SUITE-CdcrBootstrapTest-seed#[82331C9EEDA821A4]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 2235088 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[82331C9EEDA821A4]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testBootstrapWithSourceCluster
[junit4] 2> 2235089 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[82331C9EEDA821A4]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrBootstrapTest_82331C9EEDA821A4-001\cdcr-target-001
[junit4] 2> 2235089 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[82331C9EEDA821A4]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2235090 INFO (Thread-3533) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2235090 INFO (Thread-3533) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2235094 ERROR (Thread-3533) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2235190 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[82331C9EEDA821A4]) [ ] o.a.s.c.ZkTestServer start zk server on port:59863
[junit4] 2> 2235198 INFO (jetty-launcher-2502-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2235198 INFO (jetty-launcher-2502-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@cb829c{/solr,null,AVAILABLE}
[junit4] 2> 2235202 INFO (jetty-launcher-2502-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@12e0895{HTTP/1.1,[http/1.1]}{127.0.0.1:59867}
[junit4] 2> 2235202 INFO (jetty-launcher-2502-thread-1) [ ] o.e.j.s.Server Started @2237622ms
[junit4] 2> 2235202 INFO (jetty-launcher-2502-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=59867}
[junit4] 2> 2235202 ERROR (jetty-launcher-2502-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2235202 INFO (jetty-launcher-2502-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 2235202 INFO (jetty-launcher-2502-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2235202 INFO (jetty-launcher-2502-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 2235202 INFO (jetty-launcher-2502-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-01T08:21:07.586Z
[junit4] 2> 2235208 INFO (jetty-launcher-2502-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2235226 INFO (jetty-launcher-2502-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59863/solr
[junit4] 2> 2235340 INFO (jetty-launcher-2502-thread-1) [n:127.0.0.1:59867_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2235341 INFO (jetty-launcher-2502-thread-1) [n:127.0.0.1:59867_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59867_solr
[junit4] 2> 2235342 INFO (jetty-launcher-2502-thread-1) [n:127.0.0.1:59867_solr ] o.a.s.c.Overseer Overseer (id=98582794566172675-127.0.0.1:59867_solr-n_0000000000) starting
[junit4] 2> 2235354 INFO (jetty-launcher-2502-thread-1) [n:127.0.0.1:59867_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59867_solr
[junit4] 2> 2235356 INFO (zkCallback-2509-thread-1-processing-n:127.0.0.1:59867_solr) [n:127.0.0.1:59867_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2235510 INFO (jetty-launcher-2502-thread-1) [n:127.0.0.1:59867_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59867.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1417aa4
[junit4] 2> 2235518 INFO (jetty-launcher-2502-thread-1) [n:127.0.0.1:59867_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59867.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1417aa4
[junit4] 2> 2235518 INFO (jetty-launcher-2502-thread-1) [n:127.0.0.1:59867_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59867.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1417aa4
[junit4] 2> 2235519 INFO (jetty-launcher-2502-thread-1) [n:127.0.0.1:59867_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrBootstrapTest_82331C9EEDA821A4-001\cdcr-target-001\node1\.
[junit4] 1> Target zkHost = 127.0.0.1:59863/solr
[junit4] 2> 2235548 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[82331C9EEDA821A4]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrBootstrapTest_82331C9EEDA821A4-001\cdcr-source-001
[junit4] 2> 2235548 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[82331C9EEDA821A4]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 2235549 INFO (Thread-3543) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 2235549 INFO (Thread-3543) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 2235553 ERROR (Thread-3543) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2235650 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[82331C9EEDA821A4]) [ ] o.a.s.c.ZkTestServer start zk server on port:59887
[junit4] 2> 2235668 INFO (jetty-launcher-2514-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 2235669 INFO (jetty-launcher-2514-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@f41838{/solr,null,AVAILABLE}
[junit4] 2> 2235672 INFO (jetty-launcher-2514-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@414c41{HTTP/1.1,[http/1.1]}{127.0.0.1:59891}
[junit4] 2> 2235672 INFO (jetty-launcher-2514-thread-1) [ ] o.e.j.s.Server Started @2238092ms
[junit4] 2> 2235672 INFO (jetty-launcher-2514-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=59891}
[junit4] 2> 2235672 ERROR (jetty-launcher-2514-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 2235672 INFO (jetty-launcher-2514-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 2235672 INFO (jetty-launcher-2514-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 2235672 INFO (jetty-launcher-2514-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 2235672 INFO (jetty-launcher-2514-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-09-01T08:21:08.056Z
[junit4] 2> 2235682 INFO (jetty-launcher-2514-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 2235707 INFO (jetty-launcher-2514-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59887/solr
[junit4] 2> 2235836 INFO (jetty-launcher-2514-thread-1) [n:127.0.0.1:59891_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 2235837 INFO (jetty-launcher-2514-thread-1) [n:127.0.0.1:59891_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59891_solr
[junit4] 2> 2235838 INFO (jetty-launcher-2514-thread-1) [n:127.0.0.1:59891_solr ] o.a.s.c.Overseer Overseer (id=98582794596188163-127.0.0.1:59891_solr-n_0000000000) starting
[junit4] 2> 2235847 INFO (jetty-launcher-2514-thread-1) [n:127.0.0.1:59891_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59891_solr
[junit4] 2> 2235848 INFO (OverseerStateUpdate-98582794596188163-127.0.0.1:59891_solr-n_0000000000) [n:127.0.0.1:59891_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2236059 INFO (jetty-launcher-2514-thread-1) [n:127.0.0.1:59891_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59891.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1417aa4
[junit4] 2> 2236069 INFO (jetty-launcher-2514-thread-1) [n:127.0.0.1:59891_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59891.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1417aa4
[junit4] 2> 2236069 INFO (jetty-launcher-2514-thread-1) [n:127.0.0.1:59891_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59891.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1417aa4
[junit4] 2> 2236071 INFO (jetty-launcher-2514-thread-1) [n:127.0.0.1:59891_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrBootstrapTest_82331C9EEDA821A4-001\cdcr-source-001\node1\.
[junit4] 2> 2236144 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[82331C9EEDA821A4]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2236145 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[82331C9EEDA821A4]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59887/solr ready
[junit4] 2> 2236147 INFO (qtp14096031-18454) [n:127.0.0.1:59891_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=cdcr-source&name=cdcr-source&nrtReplicas=1&action=CREATE&numShards=1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2236154 INFO (OverseerThreadFactory-7747-thread-1-processing-n:127.0.0.1:59891_solr) [n:127.0.0.1:59891_solr ] o.a.s.c.CreateCollectionCmd Create collection cdcr-source
[junit4] 2> 2236262 INFO (OverseerStateUpdate-98582794596188163-127.0.0.1:59891_solr-n_0000000000) [n:127.0.0.1:59891_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"cdcr-source",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"cdcr-source_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:59891/solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 2236464 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=cdcr-source&newCollection=true&collection=cdcr-source&version=2&replicaType=NRT&coreNodeName=core_node1&name=cdcr-source_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
[junit4] 2> 2236466 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2236576 INFO (zkCallback-2521-thread-1-processing-n:127.0.0.1:59891_solr) [n:127.0.0.1:59891_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2236577 INFO (zkCallback-2521-thread-2-processing-n:127.0.0.1:59891_solr) [n:127.0.0.1:59891_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2237506 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2237516 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema.xml instead
[junit4] 2> 2237523 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.IndexSchema [cdcr-source_shard1_replica_n1] Schema name=minimal
[junit4] 2> 2237532 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2237539 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.ManagedIndexSchema Created and persisted managed schema znode at /configs/cdcr-source/managed-schema
[junit4] 2> 2237543 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, renamed the non-managed schema /configs/cdcr-source/schema.xml to /configs/cdcr-source/schema.xml.bak
[junit4] 2> 2237544 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-source_shard1_replica_n1' using configuration from collection cdcr-source, trusted=true
[junit4] 2> 2237545 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59891.solr.core.cdcr-source.shard1.replica_n1' (registry 'solr.core.cdcr-source.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1417aa4
[junit4] 2> 2237545 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2237545 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [[cdcr-source_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrBootstrapTest_82331C9EEDA821A4-001\cdcr-source-001\node1\cdcr-source_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrBootstrapTest_82331C9EEDA821A4-001\cdcr-source-001\node1\.\cdcr-source_shard1_replica_n1\data\]
[junit4] 2> 2237852 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 2237853 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2237855 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2237856 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2237863 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@174125b[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 2237865 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-source
[junit4] 2> 2237866 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-source
[junit4] 2> 2237866 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-source/managed-schema
[junit4] 2> 2237867 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 2237868 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2237883 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrBufferStateManager Created znode /collections/cdcr-source/cdcr/state/buffer
[junit4] 2> 2237893 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrProcessStateManager Created znode /collections/cdcr-source/cdcr/state/process
[junit4] 2> 2237945 INFO (searcherExecutor-7750-thread-1-processing-n:127.0.0.1:59891_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@174125b[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2237945 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577324716049301504
[junit4] 2> 2237956 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2237956 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2237956 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:59891/solr/cdcr-source_shard1_replica_n1/
[junit4] 2> 2237956 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2237957 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:59891/solr/cdcr-source_shard1_replica_n1/ has no replicas
[junit4] 2> 2237957 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2237963 INFO (zkCallback-2521-thread-2-processing-n:127.0.0.1:59891_solr) [n:127.0.0.1:59891_solr ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-source:shard1
[junit4] 2> 2237965 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:59891/solr/cdcr-source_shard1_replica_n1/ shard1
[junit4] 2> 2238080 INFO (zkCallback-2521-thread-2-processing-n:127.0.0.1:59891_solr) [n:127.0.0.1:59891_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2238081 INFO (zkCallback-2521-thread-1-processing-n:127.0.0.1:59891_solr) [n:127.0.0.1:59891_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2238124 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2238127 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=cdcr-source&newCollection=true&collection=cdcr-source&version=2&replicaType=NRT&coreNodeName=core_node1&name=cdcr-source_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=1662
[junit4] 2> 2238136 INFO (qtp14096031-18454) [n:127.0.0.1:59891_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2238157 INFO (OverseerCollectionConfigSetProcessor-98582794596188163-127.0.0.1:59891_solr-n_0000000000) [n:127.0.0.1:59891_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2238238 INFO (zkCallback-2521-thread-1-processing-n:127.0.0.1:59891_solr) [n:127.0.0.1:59891_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2238238 INFO (zkCallback-2521-thread-2-processing-n:127.0.0.1:59891_solr) [n:127.0.0.1:59891_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2239137 INFO (qtp14096031-18454) [n:127.0.0.1:59891_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=cdcr-source&name=cdcr-source&nrtReplicas=1&action=CREATE&numShards=1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2} status=0 QTime=2988
[junit4] 1> Adding 10 docs with commit=true, numDocs=100
[junit4] 2> 2239177 INFO (qtp14096031-18458) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=36
[junit4] 2> 2239179 INFO (qtp14096031-18459) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577324717343244288,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2239179 INFO (qtp14096031-18459) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@ce7a06 commitCommandVersion:1577324717343244288
[junit4] 2> 2239212 INFO (qtp14096031-18459) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1320bfb[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 2239213 INFO (qtp14096031-18459) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2239216 INFO (searcherExecutor-7750-thread-1-processing-n:127.0.0.1:59891_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@1320bfb[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100)))}
[junit4] 2> 2239216 INFO (qtp14096031-18459) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=37
[junit4] 1> Adding 10 docs with commit=true, numDocs=200
[junit4] 2> 2239239 INFO (qtp14096031-18460) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=21
[junit4] 2> 2239241 INFO (qtp14096031-18461) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577324717408256000,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2239241 INFO (qtp14096031-18461) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@ce7a06 commitCommandVersion:1577324717408256000
[junit4] 2> 2239263 INFO (qtp14096031-18461) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1396607[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 2239263 INFO (qtp14096031-18461) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2239266 INFO (searcherExecutor-7750-thread-1-processing-n:127.0.0.1:59891_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@1396607[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100)))}
[junit4] 2> 2239266 INFO (qtp14096031-18461) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=25
[junit4] 1> Adding 10 docs with commit=true, numDocs=300
[junit4] 2> 2239291 INFO (qtp14096031-18456) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=22
[junit4] 2> 2239292 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577324717461733376,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2239292 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@ce7a06 commitCommandVersion:1577324717461733376
[junit4] 2> 2239323 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1d9300d[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 2239324 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2239328 INFO (searcherExecutor-7750-thread-1-processing-n:127.0.0.1:59891_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@1d9300d[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100)))}
[junit4] 2> 2239328 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=36
[junit4] 1> Adding 10 docs with commit=true, numDocs=400
[junit4] 2> 2239353 INFO (qtp14096031-18454) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=23
[junit4] 2> 2239354 INFO (qtp14096031-18458) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577324717526745088,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2239356 INFO (qtp14096031-18458) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@ce7a06 commitCommandVersion:1577324717526745088
[junit4] 2> 2239470 INFO (qtp14096031-18458) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@4bb6e2[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 2239471 INFO (qtp14096031-18458) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2239473 INFO (searcherExecutor-7750-thread-1-processing-n:127.0.0.1:59891_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@4bb6e2[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100) Uninverting(_3(7.0.0):C100)))}
[junit4] 2> 2239473 INFO (qtp14096031-18458) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=118
[junit4] 1> Adding 10 docs with commit=true, numDocs=500
[junit4] 2> 2239495 INFO (qtp14096031-18459) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=19
[junit4] 2> 2239501 INFO (qtp14096031-18460) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577324717680885760,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2239501 INFO (qtp14096031-18460) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@ce7a06 commitCommandVersion:1577324717680885760
[junit4] 2> 2239553 INFO (qtp14096031-18460) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@71bf61[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 2239554 INFO (qtp14096031-18460) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2239557 INFO (searcherExecutor-7750-thread-1-processing-n:127.0.0.1:59891_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@71bf61[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100) Uninverting(_3(7.0.0):C100) Uninverting(_4(7.0.0):C100)))}
[junit4] 2> 2239558 INFO (qtp14096031-18460) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=58
[junit4] 1> Adding 10 docs with commit=true, numDocs=600
[junit4] 2> 2239578 INFO (qtp14096031-18461) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=17
[junit4] 2> 2239579 INFO (qtp14096031-18456) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577324717762674688,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2239580 INFO (qtp14096031-18456) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@ce7a06 commitCommandVersion:1577324717762674688
[junit4] 2> 2239622 INFO (qtp14096031-18456) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6072a3[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 2239623 INFO (qtp14096031-18456) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2239625 INFO (searcherExecutor-7750-thread-1-processing-n:127.0.0.1:59891_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@6072a3[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100) Uninverting(_3(7.0.0):C100) Uninverting(_4(7.0.0):C100) Uninverting(_5(7.0.0):C100)))}
[junit4] 2> 2239625 INFO (qtp14096031-18456) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=46
[junit4] 1> Adding 10 docs with commit=true, numDocs=700
[junit4] 2> 2239640 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=12
[junit4] 2> 2239641 INFO (qtp14096031-18454) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577324717827686400,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2239641 INFO (qtp14096031-18454) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@ce7a06 commitCommandVersion:1577324717827686400
[junit4] 2> 2239803 INFO (qtp14096031-18454) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@185d1ad[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 2239804 INFO (qtp14096031-18454) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2239816 INFO (searcherExecutor-7750-thread-1-processing-n:127.0.0.1:59891_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@185d1ad[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100) Uninverting(_3(7.0.0):C100) Uninverting(_4(7.0.0):C100) Uninverting(_5(7.0.0):C100) Uninverting(_6(7.0.0):C100)))}
[junit4] 2> 2239817 INFO (qtp14096031-18454) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=175
[junit4] 1> Adding 10 docs with commit=true, numDocs=800
[junit4] 2> 2239839 INFO (qtp14096031-18458) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=21
[junit4] 2> 2239839 INFO (qtp14096031-18459) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577324718035304459,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2239839 INFO (qtp14096031-18459) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@ce7a06 commitCommandVersion:1577324718035304459
[junit4] 2> 2239960 INFO (qtp14096031-18459) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@108874a[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 2239961 INFO (qtp14096031-18459) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2239967 INFO (searcherExecutor-7750-thread-1-processing-n:127.0.0.1:59891_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@108874a[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100) Uninverting(_3(7.0.0):C100) Uninverting(_4(7.0.0):C100) Uninverting(_5(7.0.0):C100) Uninverting(_6(7.0.0):C100) Uninverting(_7(7.0.0):C100)))}
[junit4] 2> 2239968 INFO (qtp14096031-18459) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=125
[junit4] 1> Adding 10 docs with commit=true, numDocs=900
[junit4] 2> 2240007 INFO (qtp14096031-18460) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=36
[junit4] 2> 2240013 INFO (qtp14096031-18461) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577324718217756672,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2240013 INFO (qtp14096031-18461) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@ce7a06 commitCommandVersion:1577324718217756672
[junit4] 2> 2240178 INFO (qtp14096031-18461) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@181be6b[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 2240179 INFO (qtp14096031-18461) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2240182 INFO (searcherExecutor-7750-thread-1-processing-n:127.0.0.1:59891_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@181be6b[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100) Uninverting(_3(7.0.0):C100) Uninverting(_4(7.0.0):C100) Uninverting(_5(7.0.0):C100) Uninverting(_6(7.0.0):C100) Uninverting(_7(7.0.0):C100) Uninverting(_8(7.0.0):C100)))}
[junit4] 2> 2240182 INFO (qtp14096031-18461) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=169
[junit4] 1> Adding 10 docs with commit=true, numDocs=1000
[junit4] 2> 2240206 INFO (qtp14096031-18456) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=21
[junit4] 2> 2240207 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577324718421180416,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2240207 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@ce7a06 commitCommandVersion:1577324718421180416
[junit4] 2> 2240284 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1e67cf5[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 2240285 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2240290 INFO (searcherExecutor-7750-thread-1-processing-n:127.0.0.1:59891_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@1e67cf5[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100) Uninverting(_3(7.0.0):C100) Uninverting(_4(7.0.0):C100) Uninverting(_5(7.0.0):C100) Uninverting(_6(7.0.0):C100) Uninverting(_7(7.0.0):C100) Uninverting(_8(7.0.0):C100) Uninverting(_9(7.0.0):C100)))}
[junit4] 2> 2240291 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=83
[junit4] 2> 2240294 INFO (qtp14096031-18454) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&_stateVer_=cdcr-source:4&wt=javabin&version=2} hits=1000 status=0 QTime=1
[junit4] 2> 2240342 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[82331C9EEDA821A4]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2240344 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[82331C9EEDA821A4]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59863/solr ready
[junit4] 2> 2240349 INFO (qtp24125483-18415) [n:127.0.0.1:59867_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=cdcr-target&name=cdcr-target&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 2240352 INFO (OverseerThreadFactory-7739-thread-1-processing-n:127.0.0.1:59867_solr) [n:127.0.0.1:59867_solr ] o.a.s.c.CreateCollectionCmd Create collection cdcr-target
[junit4] 2> 2240498 INFO (OverseerStateUpdate-98582794566172675-127.0.0.1:59867_solr-n_0000000000) [n:127.0.0.1:59867_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"cdcr-target",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"cdcr-target_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:59867/solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 2240701 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node1&collection.configName=cdcr-target&newCollection=true&name=cdcr-target_shard1_replica_n1&action=CREATE&numShards=1&collection=cdcr-target&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 2240703 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 2240816 INFO (zkCallback-2509-thread-1-processing-n:127.0.0.1:59867_solr) [n:127.0.0.1:59867_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2240817 INFO (zkCallback-2509-thread-2-processing-n:127.0.0.1:59867_solr) [n:127.0.0.1:59867_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2241734 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 2241741 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema.xml instead
[junit4] 2> 2241745 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.s.IndexSchema [cdcr-target_shard1_replica_n1] Schema name=minimal
[junit4] 2> 2241750 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 2241751 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.s.ManagedIndexSchema Created and persisted managed schema znode at /configs/cdcr-target/managed-schema
[junit4] 2> 2241754 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, renamed the non-managed schema /configs/cdcr-target/schema.xml to /configs/cdcr-target/schema.xml.bak
[junit4] 2> 2241754 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-target_shard1_replica_n1' using configuration from collection cdcr-target, trusted=true
[junit4] 2> 2241755 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59867.solr.core.cdcr-target.shard1.replica_n1' (registry 'solr.core.cdcr-target.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@1417aa4
[junit4] 2> 2241756 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 2241756 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.SolrCore [[cdcr-target_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrBootstrapTest_82331C9EEDA821A4-001\cdcr-target-001\node1\cdcr-target_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrBootstrapTest_82331C9EEDA821A4-001\cdcr-target-001\node1\.\cdcr-target_shard1_replica_n1\data\]
[junit4] 2> 2241848 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 2241848 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 2241850 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 2241850 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 2241851 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@b8d144[cdcr-target_shard1_replica_n1] main]
[junit4] 2> 2241853 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-target
[junit4] 2> 2241854 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-target
[junit4] 2> 2241854 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-target/managed-schema
[junit4] 2> 2241854 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 2241855 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 2241858 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.CdcrBufferStateManager Created znode /collections/cdcr-target/cdcr/state/buffer
[junit4] 2> 2241859 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.CdcrProcessStateManager Created znode /collections/cdcr-target/cdcr/state/process
[junit4] 2> 2241861 INFO (searcherExecutor-7756-thread-1-processing-n:127.0.0.1:59867_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-target_shard1_replica_n1] Registered new searcher Searcher@b8d144[cdcr-target_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 2241861 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577324720155525120
[junit4] 2> 2241868 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 2241868 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 2241868 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:59867/solr/cdcr-target_shard1_replica_n1/
[junit4] 2> 2241869 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 2241869 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:59867/solr/cdcr-target_shard1_replica_n1/ has no replicas
[junit4] 2> 2241869 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 2241872 INFO (zkCallback-2509-thread-2-processing-n:127.0.0.1:59867_solr) [n:127.0.0.1:59867_solr ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-target:shard1
[junit4] 2> 2241873 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:59867/solr/cdcr-target_shard1_replica_n1/ shard1
[junit4] 2> 2241978 INFO (zkCallback-2509-thread-2-processing-n:127.0.0.1:59867_solr) [n:127.0.0.1:59867_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2241978 INFO (zkCallback-2509-thread-1-processing-n:127.0.0.1:59867_solr) [n:127.0.0.1:59867_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2242025 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 2242029 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node1&collection.configName=cdcr-target&newCollection=true&name=cdcr-target_shard1_replica_n1&action=CREATE&numShards=1&collection=cdcr-target&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=1327
[junit4] 2> 2242036 INFO (qtp24125483-18415) [n:127.0.0.1:59867_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 2242138 INFO (zkCallback-2509-thread-2-processing-n:127.0.0.1:59867_solr) [n:127.0.0.1:59867_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2242138 INFO (zkCallback-2509-thread-1-processing-n:127.0.0.1:59867_solr) [n:127.0.0.1:59867_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 2242355 INFO (OverseerCollectionConfigSetProcessor-98582794566172675-127.0.0.1:59867_solr-n_0000000000) [n:127.0.0.1:59867_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 2243037 INFO (qtp24125483-18415) [n:127.0.0.1:59867_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=cdcr-target&name=cdcr-target&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=2687
[junit4] 2> 2243043 INFO (qtp24125483-18418) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&_stateVer_=cdcr-target:4&action=start&wt=javabin&version=2} status=0 QTime=4
[junit4] 2> 2243043 INFO (zkCallback-2509-thread-1-processing-n:127.0.0.1:59867_solr) [n:127.0.0.1:59867_solr ] o.a.s.h.CdcrProcessStateManager The CDCR process state has changed: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/cdcr/state/process @ cdcr-target:shard1
[junit4] 2> 2243044 INFO (zkCallback-2509-thread-1-processing-n:127.0.0.1:59867_solr) [n:127.0.0.1:59867_solr ] o.a.s.h.CdcrProcessStateManager Received new CDCR process state from watcher: STARTED @ cdcr-target:shard1
[junit4] 2> 2243051 INFO (qtp14096031-18458) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 2243053 INFO (qtp14096031-18458) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59863/solr ready
[junit4] 2> 2243064 INFO (qtp24125483-18416) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
[junit4] 2> 2243064 INFO (qtp24125483-18416) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 2243065 INFO (qtp24125483-18420) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/cdcr params={_stateVer_=cdcr-target:4&action=COLLECTIONCHECKPOINT&wt=javabin&version=2} status=0 QTime=10
[junit4] 2> 2243065 INFO (qtp14096031-18458) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Create new update log reader for target cdcr-target with checkpoint -1 @ cdcr-source:shard1
[junit4] 2> 2243065 INFO (qtp14096031-18458) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-target, shard: shard1
[junit4] 2> 2243070 INFO (qtp14096031-18458) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Submitting bootstrap task to executor
[junit4] 2> 2243071 INFO (zkCallback-2521-thread-2-processing-n:127.0.0.1:59891_solr) [n:127.0.0.1:59891_solr ] o.a.s.h.CdcrProcessStateManager The CDCR process state has changed: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/cdcr/state/process @ cdcr-source:shard1
[junit4] 2> 2243072 INFO (qtp14096031-18458) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&_stateVer_=cdcr-source:4&action=start&wt=javabin&version=2} status=0 QTime=28
[junit4] 2> 2243072 INFO (zkCallback-2521-thread-2-processing-n:127.0.0.1:59891_solr) [n:127.0.0.1:59891_solr ] o.a.s.h.CdcrProcessStateManager Received new CDCR process state from watcher: STARTED @ cdcr-source:shard1
[junit4] 2> 2243073 INFO (cdcr-bootstrap-status-2533-thread-1-processing-n:127.0.0.1:59891_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-target shard: shard1 leader: http://127.0.0.1:59867/solr/cdcr-target_shard1_replica_n1/
[junit4] 2> 2243075 INFO (qtp14096031-18459) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&_stateVer_=cdcr-source:4&action=queues&wt=javabin&version=2} status=0 QTime=1
[junit4] 1> Cdcr queue response: {responseHeader={status=0,QTime=1},queues={127.0.0.1:59863/solr={cdcr-target={queueSize=1010,lastTimestamp=}}},tlogTotalSize=47659,tlogTotalCount=10,updateLogSynchronizer=stopped}
[junit4] 2> 2243078 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&masterUrl=http://127.0.0.1:59891/solr/cdcr-source_shard1_replica_n1/&action=BOOTSTRAP&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 2243079 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577324721432690688,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2243079 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 2243081 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2243081 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-target:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=2
[junit4] 2> 2243082 INFO (qtp24125483-18418) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&action=BOOTSTRAP_STATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2243082 WARN (cdcr-bootstrap-status-2533-thread-1-processing-n:127.0.0.1:59891_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Bootstrap process was not found on target collection: cdcr-target shard: shard1, leader: http://127.0.0.1:59867/solr/cdcr-target_shard1_replica_n1/
[junit4] 2> 2243083 INFO (qtp24125483-18417) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&_stateVer_=cdcr-target:4&wt=javabin&version=2} hits=0 status=0 QTime=0
[junit4] 2> 2243083 INFO (cdcr-bootstrap-status-2533-thread-1-processing-n:127.0.0.1:59891_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-target shard: shard1 leader: http://127.0.0.1:59867/solr/cdcr-target_shard1_replica_n1/
[junit4] 2> 2243084 INFO (qtp24125483-18418) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&masterUrl=http://127.0.0.1:59891/solr/cdcr-source_shard1_replica_n1/&action=BOOTSTRAP&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2243085 INFO (qtp24125483-18413) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&action=BOOTSTRAP_STATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 2243086 INFO (cdcr-bootstrap-status-2533-thread-1-processing-n:127.0.0.1:59891_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager CDCR bootstrap running for 1 seconds, sleeping for 2000 ms
[junit4] 2> 2243089 INFO (recoveryExecutor-2507-thread-1-processing-n:127.0.0.1:59867_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 2243092 INFO (qtp14096031-18461) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577324721446322176,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 2243092 INFO (qtp14096031-18461) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@ce7a06 commitCommandVersion:1577324721446322176
[junit4] 2> 2243102 INFO (qtp14096031-18461) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@c77c2e[cdcr-source_shard1_replica_n1] realtime]
[junit4] 2> 2243103 INFO (qtp14096031-18461) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 2243103 INFO (qtp14096031-18461) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&commit_end_point=true&wt=javabin&version=2} status=0 QTime=11
[junit4] 2> 2243105 INFO (qtp14096031-18454) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 2243106 INFO (recoveryExecutor-2507-thread-1-processing-n:127.0.0.1:59867_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.IndexFetcher Master's generation: 12
[junit4] 2> 2243106 INFO (recoveryExecutor-2507-thread-1-processing-n:127.0.0.1:59867_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.IndexFetcher Master's version: 1504254075476
[junit4] 2> 2243106 INFO (recoveryExecutor-2507-thread-1-processing-n:127.0.0.1:59867_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 2243106 INFO (recoveryExecutor-2507-thread-1-processing-n:127.0.0.1:59867_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 2243106 INFO (recoveryExecutor-2507-thread-1-processing-n:127.0.0.1:59867_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.IndexFetcher Starting replication process
[junit4] 2> 2243115 INFO (qtp14096031-18458) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.ReplicationHandler Adding tlog files to list: [{size=4657, name=tlog.0000000000000000000.1577324717304446976}, {size=4778, name=tlog.0000000000000000001.1577324717384138752}, {size=4778, name=tlog.0000000000000000002.1577324717436567552}, {size=4778, name=tlog.0000000000000000003.1577324717502627840}, {size=4778, name=tlog.0000000000000000004.1577324717653622784}, {size=4778, name=tlog.0000000000000000005.1577324717743800320}, {size=4778, name=tlog.0000000000000000006.1577324717814054912}, {size=4778, name=tlog.0000000000000000007.1577324718015381504}, {size=4778, name=tlog.0000000000000000008.1577324718173716480}, {size=4778, name=tlog.0000000000000000009.1577324718398111744}]
[junit4] 2> 2243115 INFO (qtp14096031-18458) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&wt=javabin&version=2&command=filelist} status=0 QTime=8
[junit4] 2> 2243115 INFO (recoveryExecutor-2507-thread-1-processing-n:127.0.0.1:59867_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.IndexFetcher Number of files in latest index in master: 13
[junit4] 2> 2243115 INFO (recoveryExecutor-2507-thread-1-processing-n:127.0.0.1:59867_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.IndexFetcher Number of tlog files in master: 10
[junit4] 2> 2243117 INFO (recoveryExecutor-2507-thread-1-processing-n:127.0.0.1:59867_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 2243119 INFO (recoveryExecutor-2507-thread-1-processing-n:127.0.0.1:59867_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MockDirectoryWrapper(RAMDirectory@18df603 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@114f6)
[junit4] 2> 2243120 INFO (qtp14096031-18459) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2243121 INFO (qtp14096031-18460) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2243125 INFO (qtp14096031-18461) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2243127 INFO (qtp14096031-18461) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2243128 INFO (qtp14096031-18461) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_Lucene70_0.dvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2243130 INFO (qtp14096031-18456) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2243131 INFO (qtp14096031-18456) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2243133 INFO (qtp14096031-18454) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2243134 INFO (qtp14096031-18458) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.nvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2243135 INFO (qtp14096031-18459) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.fnm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2243137 INFO (qtp14096031-18460) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2243138 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_Lucene70_0.dvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2243140 INFO (qtp14096031-18461) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=segments_c&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2243141 INFO (recoveryExecutor-2507-thread-1-processing-n:127.0.0.1:59867_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:59867_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.IndexFetcher Starting download of tlog files from master: [{size=4657, name=tlog.0000000000000000000.1577324717304446976}, {size=4778, name=tlog.0000000000000000001.1577324717384138752}, {size=4778, name=tlog.0000000000000000002.1577324717436567552}, {size=4778, name=tlog.0000000000000000003.1577324717502627840}, {size=4778, name=tlog.0000000000000000004.1577324717653622784}, {size=4778, name=tlog.0000000000000000005.1577324717743800320}, {size=4778, name=tlog.0000000000000000006.1577324717814054912}, {size=4778, name=tlog.0000000000000000007.1577324718015381504}, {size=4778, name=tlog.0000000000000000008.1577324718173716480}, {size=4778, name=tlog.0000000000000000009.1577324718398111744}]
[junit4] 2> 2243152 INFO (qtp14096031-18456) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000000.1577324717304446976&wt=filestream&command=filecontent} status=0 QTime=4
[junit4] 2> 2243155 INFO (qtp14096031-18454) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000001.1577324717384138752&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2243158 INFO (qtp14096031-18458) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000002.1577324717436567552&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2243162 INFO (qtp14096031-18459) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000003.1577324717502627840&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2243164 INFO (qtp14096031-18460) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000004.1577324717653622784&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2243168 INFO (qtp14096031-18457) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000005.1577324717743800320&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2243172 INFO (qtp14096031-18461) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000006.1577324717814054912&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2243175 INFO (qtp14096031-18461) [n:127.0.0.1:59891_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000007.1577324718015381504&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 2243177 INFO (qtp14096031-18454) [n:127.0.0.
[...truncated too long message...]
d1.replica_n1, tag=1048607
[junit4] 2> 2389688 INFO (coreCloseExecutor-7890-thread-1) [n:127.0.0.1:60390_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@f421d: rootName = solr_60390, domain = solr.core.cdcr-target.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.cdcr-target.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@89d2b
[junit4] 2> 2389696 INFO (coreCloseExecutor-7890-thread-1) [n:127.0.0.1:60390_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.cdcr-target.shard1.leader, tag=1048607
[junit4] 2> 2389697 INFO (coreCloseExecutor-7890-thread-1) [n:127.0.0.1:60390_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Solr core is being closed - shutting down CDCR handler @ cdcr-target:shard1
[junit4] 2> 2389698 INFO (jetty-closer-2572-thread-1) [ ] o.a.s.c.Overseer Overseer (id=98582803928055811-127.0.0.1:60390_solr-n_0000000000) closing
[junit4] 2> 2389699 INFO (OverseerStateUpdate-98582803928055811-127.0.0.1:60390_solr-n_0000000000) [n:127.0.0.1:60390_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:60390_solr
[junit4] 2> 2389699 WARN (zkCallback-2578-thread-1-processing-n:127.0.0.1:60390_solr) [n:127.0.0.1:60390_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 2389700 INFO (jetty-closer-2572-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@1a0548d{/solr,null,UNAVAILABLE}
[junit4] 2> 2389701 ERROR (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[82331C9EEDA821A4]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 2389701 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[82331C9EEDA821A4]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:60386 60386
[junit4] 2> 2389703 INFO (Thread-3583) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:60386 60386
[junit4] 2> 2389703 WARN (Thread-3583) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2> 2 /solr/configs/cdcr-target/managed-schema
[junit4] 2> 2 /solr/collections/cdcr-target/leaders/shard1/leader
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/collections/cdcr-target/state.json
[junit4] 2> 3 /solr/collections/cdcr-target/cdcr/state/process
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 2 /solr/collections/cdcr-target/cdcr/state/buffer
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/live_nodes
[junit4] 2> 3 /solr/collections
[junit4] 2>
[junit4] 2> 2389704 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[82331C9EEDA821A4]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testConvertClusterToCdcrAndBootstrap
[junit4] 2> NOTE: leaving temporary files on disk at: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrBootstrapTest_82331C9EEDA821A4-001
[junit4] 2> Sep 01, 2017 8:23:42 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 6 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=true): {}, locale=mt, timezone=America/Asuncion
[junit4] 2> NOTE: Windows 10 10.0 x86/Oracle Corporation 1.8.0_144 (32-bit)/cpus=3,threads=1,free=203034832,total=467927040
[junit4] 2> NOTE: All tests run in this JVM: [TestSchemaSimilarityResource, DOMUtilTest, TestPushWriter, JvmMetricsTest, OverseerStatusTest, SolrSlf4jReporterTest, TestTolerantUpdateProcessorRandomCloud, TestImplicitCoreProperties, TestManagedResource, HdfsDirectoryFactoryTest, TestSimpleQParserPlugin, TestPivotHelperCode, TestCloudRecovery, AssignTest, TestInPlaceUpdatesDistrib, BasicAuthStandaloneTest, TestSolrJ, HdfsThreadLeakTest, TestQueryUtils, LeaderInitiatedRecoveryOnCommitTest, TestDocSet, TestSortByMinMaxFunction, QueryEqualityTest, PeerSyncReplicationTest, ZkNodePropsTest, FileBasedSpellCheckerTest, SolrPluginUtilsTest, SolrGangliaReporterTest, TestTlogReplica, PluginInfoTest, SaslZkACLProviderTest, JSONWriterTest, PreAnalyzedFieldTest, TestRestoreCore, TestSuggestSpellingConverter, TestRebalanceLeaders, TestBackupRepositoryFactory, SchemaVersionSpecificBehaviorTest, TestExportWriter, TestCollationFieldDocValues, PingRequestHandlerTest, TestSystemIdResolver, TestAnalyzeInfixSuggestions, FieldMutatingUpdateProcessorTest, MetricUtilsTest, CoreMergeIndexesAdminHandlerTest, ConcurrentDeleteAndCreateCollectionTest, CheckHdfsIndexTest, CdcrRequestHandlerTest, DistributedFacetPivotLargeTest, CdcrReplicationDistributedZkTest, TestDocTermOrds, DistributedExpandComponentTest, DocumentAnalysisRequestHandlerTest, TestOnReconnectListenerSupport, DistributedMLTComponentTest, JavabinLoaderTest, SolrCoreCheckLockOnStartupTest, SearchHandlerTest, TestLeaderElectionWithEmptyReplica, TestIndexSearcher, ZkStateReaderTest, TestDocumentBuilder, TestPerFieldSimilarity, BasicDistributedZk2Test, HdfsBasicDistributedZk2Test, TestSolrQueryParser, SolrCmdDistributorTest, TestRealTimeGet, PolyFieldTest, TestComplexPhraseQParserPlugin, TestBM25SimilarityFactory, TestDynamicFieldCollectionResource, TestClassNameShortening, TestCloudNestedDocsSort, DistributedQueueTest, TestRandomFlRTGCloud, RollingRestartTest, SimpleCollectionCreateDeleteTest, LeaderElectionIntegrationTest, TestWordDelimiterFilterFactory, SpellPossibilityIteratorTest, BasicZkTest, TestLockTree, OverseerModifyCollectionTest, TestFastWriter, TestBulkSchemaConcurrent, TestSolrCLIRunExample, TestSolrCloudWithSecureImpersonation, V2ApiIntegrationTest, SolrIndexMetricsTest, TestExceedMaxTermLength, TestBadConfig, TestFastOutputStream, SpellingQueryConverterTest, ConfigureRecoveryStrategyTest, HdfsRestartWhileUpdatingTest, CloudExitableDirectoryReaderTest, TestLuceneMatchVersion, TestSSLRandomization, SOLR749Test, TestInitParams, TestShortCircuitedRequests, TestCollationField, RemoteQueryErrorTest, TestRandomCollapseQParserPlugin, TestDistributedMissingSort, SuggestComponentTest, BadComponentTest, ExternalFileFieldSortTest, DistributedFacetPivotWhiteBoxTest, ReplicaListTransformerTest, HdfsCollectionsAPIDistributedZkTest, TestPartialUpdateDeduplication, TestFileDictionaryLookup, SuggesterFSTTest, UpdateRequestProcessorFactoryTest, SimpleMLTQParserTest, TestReloadDeadlock, StressHdfsTest, TestPerFieldSimilarityWithDefaultOverride, HdfsDirectoryTest, CollectionStateFormat2Test, FileUtilsTest, CopyFieldTest, TestInitQParser, BooleanFieldTest, TestReplicationHandlerBackup, TestCrossCoreJoin, BadIndexSchemaTest, SolrIndexConfigTest, TestFiltering, ManagedSchemaRoundRobinCloudTest, TestGraphTermsQParserPlugin, TestStressUserVersions, SecurityConfHandlerTest, BlockJoinFacetSimpleTest, AutoScalingHandlerTest, TestSerializedLuceneMatchVersion, TestRandomFaceting, CollectionsAPIAsyncDistributedZkTest, TestRequestForwarding, SynonymTokenizerTest, EchoParamsTest, ConvertedLegacyTest, BlockDirectoryTest, DistributedTermsComponentTest, TestMacroExpander, TermsComponentTest, TestManagedSchema, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestSolrDeletionPolicy1, TestSchemaManager, TestNumericTokenStream, TestJettySolrRunner, TestSolrCoreSnapshots, DocValuesNotIndexedTest, PreAnalyzedUpdateProcessorTest, TestExclusionRuleCollectionAccess, UninvertDocValuesMergePolicyTest, LeaderFailoverAfterPartitionTest, SystemInfoHandlerTest, TestExpandComponent, UpdateParamsTest, SolrCloudExampleTest, TolerantUpdateProcessorTest, DataDrivenBlockJoinTest, TestSolr4Spatial, TestSolrCloudWithHadoopAuthPlugin, SearchHandlerTest, ShardRoutingCustomTest, TestLRUStatsCache, UtilsToolTest, TestGraphMLResponseWriter, MergeStrategyTest, DocValuesMissingTest, TestCryptoKeys, TestConfigReload, TestDistributedGrouping, BasicDistributedZkTest, OpenCloseCoreStressTest, OverseerTest, FullSolrCloudDistribCmdsTest, TestDistributedSearch, TermVectorComponentDistributedTest, TestRangeQuery, StatsComponentTest, PeerSyncTest, HighlighterTest, DistributedQueryElevationComponentTest, SignatureUpdateProcessorFactoryTest, SpellCheckCollatorTest, SolrRequestParserTest, DocValuesTest, StatelessScriptUpdateProcessorFactoryTest, SolrInfoBeanTest, CacheHeaderTest, LukeRequestHandlerTest, TestQueryTypes, DocumentBuilderTest, TermVectorComponentTest, FastVectorHighlighterTest, LoggingHandlerTest, TestJmxIntegration, QueryParsingTest, JsonLoaderTest, TestComponentsName, TestStressRecovery, SampleTest, TestBinaryField, TestSearchPerf, MinimalSchemaTest, TestConfig, TestPostingsSolrHighlighter, TestCodecSupport, TimeZoneUtilsTest, OpenExchangeRatesOrgProviderTest, DateMathParserTest, RAMDirectoryFactoryTest, ClusterStateTest, TestLRUCache, TestUtils, SliceStateTest, CircularListTest, TestRTGBase, DistributedIntervalFacetingTest, SolrTestCaseJ4Test, TestSimpleTrackingShardHandler, TestTolerantSearch, TestEmbeddedSolrServerConstructors, TestEmbeddedSolrServerSchemaAPI, ConnectionReuseTest, ActionThrottleTest, AliasIntegrationTest, AsyncCallRequestStatusResponseTest, BaseCdcrDistributedZkTest, CdcrBootstrapTest]
[junit4] Completed [502/729 (1!)] on J1 in 155.65s, 3 tests, 1 failure <<< FAILURES!
[...truncated 40161 lines...]
[JENKINS] Lucene-Solr-7.0-Windows (32bit/jdk1.8.0_144) - Build #
124 - Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.0-Windows/124/
Java: 32bit/jdk1.8.0_144 -server -XX:+UseParallelGC
1 tests failed.
FAILED: org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster
Error Message:
Document mismatch on target after sync expected:<2000> but was:<1100>
Stack Trace:
java.lang.AssertionError: Document mismatch on target after sync expected:<2000> but was:<1100>
at __randomizedtesting.SeedInfo.seed([5793E6163EF30DD:D13C753884B98326]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster(CdcrBootstrapTest.java:309)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 12637 lines...]
[junit4] Suite: org.apache.solr.cloud.CdcrBootstrapTest
[junit4] 2> Creating dataDir: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrBootstrapTest_5793E6163EF30DD-001\init-core-data-001
[junit4] 2> 1724000 INFO (SUITE-CdcrBootstrapTest-seed#[5793E6163EF30DD]-worker) [ ] o.a.s.SolrTestCaseJ4 Using TrieFields (NUMERIC_POINTS_SYSPROP=false) w/NUMERIC_DOCVALUES_SYSPROP=true
[junit4] 2> 1724001 INFO (SUITE-CdcrBootstrapTest-seed#[5793E6163EF30DD]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (true) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 1724002 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testConvertClusterToCdcrAndBootstrap
[junit4] 2> 1724003 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrBootstrapTest_5793E6163EF30DD-001\cdcr-target-001
[junit4] 2> 1724003 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1724003 INFO (Thread-3837) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1724003 INFO (Thread-3837) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1724007 ERROR (Thread-3837) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1724103 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.c.ZkTestServer start zk server on port:58227
[junit4] 2> 1724112 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e3a1987c20000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1724114 INFO (jetty-launcher-2624-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1724114 INFO (jetty-launcher-2624-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1630bc{/solr,null,AVAILABLE}
[junit4] 2> 1724115 INFO (jetty-launcher-2624-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@609722{HTTP/1.1,[http/1.1]}{127.0.0.1:58231}
[junit4] 2> 1724116 INFO (jetty-launcher-2624-thread-1) [ ] o.e.j.s.Server Started @1729742ms
[junit4] 2> 1724116 INFO (jetty-launcher-2624-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=58231}
[junit4] 2> 1724116 ERROR (jetty-launcher-2624-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1724116 INFO (jetty-launcher-2624-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 1724116 INFO (jetty-launcher-2624-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1724116 INFO (jetty-launcher-2624-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 1724116 INFO (jetty-launcher-2624-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-31T21:01:45.392Z
[junit4] 2> 1724117 INFO (jetty-launcher-2624-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1724133 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e3a1987c20001, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1724137 INFO (jetty-launcher-2624-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58227/solr
[junit4] 2> 1724208 INFO (jetty-launcher-2624-thread-1) [n:127.0.0.1:58231_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1724208 INFO (jetty-launcher-2624-thread-1) [n:127.0.0.1:58231_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:58231_solr
[junit4] 2> 1724209 INFO (jetty-launcher-2624-thread-1) [n:127.0.0.1:58231_solr ] o.a.s.c.Overseer Overseer (id=98580123174961155-127.0.0.1:58231_solr-n_0000000000) starting
[junit4] 2> 1724214 INFO (jetty-launcher-2624-thread-1) [n:127.0.0.1:58231_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58231_solr
[junit4] 2> 1724222 INFO (zkCallback-2631-thread-1-processing-n:127.0.0.1:58231_solr) [n:127.0.0.1:58231_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1724334 INFO (jetty-launcher-2624-thread-1) [n:127.0.0.1:58231_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58231.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19390db
[junit4] 2> 1724339 INFO (jetty-launcher-2624-thread-1) [n:127.0.0.1:58231_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58231.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19390db
[junit4] 2> 1724340 INFO (jetty-launcher-2624-thread-1) [n:127.0.0.1:58231_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58231.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19390db
[junit4] 2> 1724341 INFO (jetty-launcher-2624-thread-1) [n:127.0.0.1:58231_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrBootstrapTest_5793E6163EF30DD-001\cdcr-target-001\node1\.
[junit4] 2> 1724360 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e3a1987c20004, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 1> Target zkHost = 127.0.0.1:58227/solr
[junit4] 2> 1724368 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrBootstrapTest_5793E6163EF30DD-001\cdcr-source-001
[junit4] 2> 1724368 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 1724368 INFO (Thread-3847) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 1724368 INFO (Thread-3847) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 1724372 ERROR (Thread-3847) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1724468 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.c.ZkTestServer start zk server on port:58251
[junit4] 2> 1724482 INFO (jetty-launcher-2636-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1724483 INFO (jetty-launcher-2636-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@198b5b0{/solr,null,AVAILABLE}
[junit4] 2> 1724485 INFO (jetty-launcher-2636-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1f2f92e{HTTP/1.1,[http/1.1]}{127.0.0.1:58255}
[junit4] 2> 1724486 INFO (jetty-launcher-2636-thread-1) [ ] o.e.j.s.Server Started @1730112ms
[junit4] 2> 1724486 INFO (jetty-launcher-2636-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=58255}
[junit4] 2> 1724486 ERROR (jetty-launcher-2636-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1724486 INFO (jetty-launcher-2636-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 1724486 INFO (jetty-launcher-2636-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1724486 INFO (jetty-launcher-2636-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 1724486 INFO (jetty-launcher-2636-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-31T21:01:45.762Z
[junit4] 2> 1724492 INFO (jetty-launcher-2636-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1724512 INFO (jetty-launcher-2636-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58251/solr
[junit4] 2> 1724649 INFO (jetty-launcher-2636-thread-1) [n:127.0.0.1:58255_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1724650 INFO (jetty-launcher-2636-thread-1) [n:127.0.0.1:58255_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:58255_solr
[junit4] 2> 1724650 INFO (jetty-launcher-2636-thread-1) [n:127.0.0.1:58255_solr ] o.a.s.c.Overseer Overseer (id=98580123198881795-127.0.0.1:58255_solr-n_0000000000) starting
[junit4] 2> 1724656 INFO (jetty-launcher-2636-thread-1) [n:127.0.0.1:58255_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58255_solr
[junit4] 2> 1724657 INFO (zkCallback-2643-thread-1-processing-n:127.0.0.1:58255_solr) [n:127.0.0.1:58255_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1724854 INFO (jetty-launcher-2636-thread-1) [n:127.0.0.1:58255_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58255.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19390db
[junit4] 2> 1724859 INFO (jetty-launcher-2636-thread-1) [n:127.0.0.1:58255_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58255.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19390db
[junit4] 2> 1724859 INFO (jetty-launcher-2636-thread-1) [n:127.0.0.1:58255_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58255.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19390db
[junit4] 2> 1724860 INFO (jetty-launcher-2636-thread-1) [n:127.0.0.1:58255_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrBootstrapTest_5793E6163EF30DD-001\cdcr-source-001\node1\.
[junit4] 2> 1724882 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e3a19892f0004, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1724894 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e3a19892f0006, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1724897 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1724898 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:58251/solr ready
[junit4] 2> 1724900 INFO (qtp6051127-17268) [n:127.0.0.1:58255_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=cdcr-source&name=cdcr-source&nrtReplicas=1&action=CREATE&numShards=1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1724902 INFO (OverseerThreadFactory-8775-thread-1-processing-n:127.0.0.1:58255_solr) [n:127.0.0.1:58255_solr ] o.a.s.c.CreateCollectionCmd Create collection cdcr-source
[junit4] 2> 1725009 INFO (OverseerStateUpdate-98580123198881795-127.0.0.1:58255_solr-n_0000000000) [n:127.0.0.1:58255_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"cdcr-source",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"cdcr-source_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:58255/solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 1725213 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=cdcr-source&newCollection=true&collection=cdcr-source&version=2&replicaType=NRT&coreNodeName=core_node1&name=cdcr-source_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
[junit4] 2> 1725214 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1725323 INFO (zkCallback-2643-thread-1-processing-n:127.0.0.1:58255_solr) [n:127.0.0.1:58255_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1725325 INFO (zkCallback-2643-thread-2-processing-n:127.0.0.1:58255_solr) [n:127.0.0.1:58255_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1726232 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1726237 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.IndexSchema [cdcr-source_shard1_replica_n1] Schema name=minimal
[junit4] 2> 1726240 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1726240 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-source_shard1_replica_n1' using configuration from collection cdcr-source, trusted=true
[junit4] 2> 1726241 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58255.solr.core.cdcr-source.shard1.replica_n1' (registry 'solr.core.cdcr-source.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19390db
[junit4] 2> 1726241 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1726241 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [[cdcr-source_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrBootstrapTest_5793E6163EF30DD-001\cdcr-source-001\node1\cdcr-source_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrBootstrapTest_5793E6163EF30DD-001\cdcr-source-001\node1\.\cdcr-source_shard1_replica_n1\data\]
[junit4] 2> 1726324 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
[junit4] 2> 1726324 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1726329 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1726329 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1726332 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@2cbfa1[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 1726334 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-source
[junit4] 2> 1726334 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-source
[junit4] 2> 1726335 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1726348 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577281973255143424
[junit4] 2> 1726350 INFO (searcherExecutor-8778-thread-1-processing-n:127.0.0.1:58255_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@2cbfa1[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 1726355 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1726355 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1726355 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:58255/solr/cdcr-source_shard1_replica_n1/
[junit4] 2> 1726356 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1726356 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:58255/solr/cdcr-source_shard1_replica_n1/ has no replicas
[junit4] 2> 1726356 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1726360 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:58255/solr/cdcr-source_shard1_replica_n1/ shard1
[junit4] 2> 1726465 INFO (zkCallback-2643-thread-2-processing-n:127.0.0.1:58255_solr) [n:127.0.0.1:58255_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1726465 INFO (zkCallback-2643-thread-1-processing-n:127.0.0.1:58255_solr) [n:127.0.0.1:58255_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1726513 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1726514 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=cdcr-source&newCollection=true&collection=cdcr-source&version=2&replicaType=NRT&coreNodeName=core_node1&name=cdcr-source_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=1301
[junit4] 2> 1726517 INFO (qtp6051127-17268) [n:127.0.0.1:58255_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 1726618 INFO (zkCallback-2643-thread-1-processing-n:127.0.0.1:58255_solr) [n:127.0.0.1:58255_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1726618 INFO (zkCallback-2643-thread-2-processing-n:127.0.0.1:58255_solr) [n:127.0.0.1:58255_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1726903 INFO (OverseerCollectionConfigSetProcessor-98580123198881795-127.0.0.1:58255_solr-n_0000000000) [n:127.0.0.1:58255_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 1727517 INFO (qtp6051127-17268) [n:127.0.0.1:58255_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=cdcr-source&name=cdcr-source&nrtReplicas=1&action=CREATE&numShards=1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2} status=0 QTime=2617
[junit4] 1> Adding 10 docs with commit=true, numDocs=100
[junit4] 2> 1727530 INFO (qtp6051127-17271) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2}{add=[source_0 (1577281974486171648), source_1 (1577281974491414528), source_2 (1577281974491414529), source_3 (1577281974491414530), source_4 (1577281974491414531), source_5 (1577281974491414532), source_6 (1577281974491414533), source_7 (1577281974491414534), source_8 (1577281974491414535), source_9 (1577281974491414536), ... (100 adds)]} 0 8
[junit4] 2> 1727531 INFO (qtp6051127-17272) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577281974495608832,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1727531 INFO (qtp6051127-17272) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@179747f commitCommandVersion:1577281974495608832
[junit4] 2> 1727573 INFO (qtp6051127-17272) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@a3f3ae[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 1727573 INFO (qtp6051127-17272) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1727574 INFO (searcherExecutor-8778-thread-1-processing-n:127.0.0.1:58255_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@a3f3ae[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100)))}
[junit4] 2> 1727574 INFO (qtp6051127-17272) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 42
[junit4] 1> Adding 10 docs with commit=true, numDocs=200
[junit4] 2> 1727579 INFO (qtp6051127-17267) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2}{add=[source_100 (1577281974541746176), source_101 (1577281974544891904), source_102 (1577281974544891905), source_103 (1577281974544891906), source_104 (1577281974544891907), source_105 (1577281974544891908), source_106 (1577281974544891909), source_107 (1577281974544891910), source_108 (1577281974544891911), source_109 (1577281974544891912), ... (100 adds)]} 0 4
[junit4] 2> 1727580 INFO (qtp6051127-17273) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577281974546989056,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1727580 INFO (qtp6051127-17273) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@179747f commitCommandVersion:1577281974546989056
[junit4] 2> 1727605 INFO (qtp6051127-17273) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1f1368d[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 1727605 INFO (qtp6051127-17273) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1727606 INFO (searcherExecutor-8778-thread-1-processing-n:127.0.0.1:58255_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@1f1368d[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100)))}
[junit4] 2> 1727606 INFO (qtp6051127-17273) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 26
[junit4] 1> Adding 10 docs with commit=true, numDocs=300
[junit4] 2> 1727610 INFO (qtp6051127-17270) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2}{add=[source_200 (1577281974575300608), source_201 (1577281974577397760), source_202 (1577281974577397761), source_203 (1577281974577397762), source_204 (1577281974577397763), source_205 (1577281974577397764), source_206 (1577281974577397765), source_207 (1577281974577397766), source_208 (1577281974577397767), source_209 (1577281974577397768), ... (100 adds)]} 0 3
[junit4] 2> 1727611 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577281974579494912,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1727611 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@179747f commitCommandVersion:1577281974579494912
[junit4] 2> 1727635 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1ef6c08[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 1727636 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1727637 INFO (searcherExecutor-8778-thread-1-processing-n:127.0.0.1:58255_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@1ef6c08[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100)))}
[junit4] 2> 1727637 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 25
[junit4] 1> Adding 10 docs with commit=true, numDocs=400
[junit4] 2> 1727640 INFO (qtp6051127-17268) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2}{add=[source_300 (1577281974607806464), source_301 (1577281974608855040), source_302 (1577281974608855041), source_303 (1577281974608855042), source_304 (1577281974608855043), source_305 (1577281974608855044), source_306 (1577281974608855045), source_307 (1577281974608855046), source_308 (1577281974608855047), source_309 (1577281974608855048), ... (100 adds)]} 0 2
[junit4] 2> 1727641 INFO (qtp6051127-17271) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577281974610952192,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1727641 INFO (qtp6051127-17271) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@179747f commitCommandVersion:1577281974610952192
[junit4] 2> 1727666 INFO (qtp6051127-17271) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@15b501b[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 1727666 INFO (qtp6051127-17271) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1727667 INFO (searcherExecutor-8778-thread-1-processing-n:127.0.0.1:58255_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@15b501b[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100) Uninverting(_3(7.0.0):C100)))}
[junit4] 2> 1727667 INFO (qtp6051127-17271) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 25
[junit4] 1> Adding 10 docs with commit=true, numDocs=500
[junit4] 2> 1727670 INFO (qtp6051127-17272) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2}{add=[source_400 (1577281974639263744), source_401 (1577281974640312320), source_402 (1577281974640312321), source_403 (1577281974640312322), source_404 (1577281974640312323), source_405 (1577281974640312324), source_406 (1577281974640312325), source_407 (1577281974640312326), source_408 (1577281974640312327), source_409 (1577281974640312328), ... (100 adds)]} 0 2
[junit4] 2> 1727671 INFO (qtp6051127-17267) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577281974642409472,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1727671 INFO (qtp6051127-17267) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@179747f commitCommandVersion:1577281974642409472
[junit4] 2> 1727698 INFO (qtp6051127-17267) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@a5c33e[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 1727698 INFO (qtp6051127-17267) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1727699 INFO (searcherExecutor-8778-thread-1-processing-n:127.0.0.1:58255_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@a5c33e[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100) Uninverting(_3(7.0.0):C100) Uninverting(_4(7.0.0):C100)))}
[junit4] 2> 1727699 INFO (qtp6051127-17267) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 27
[junit4] 1> Adding 10 docs with commit=true, numDocs=600
[junit4] 2> 1727703 INFO (qtp6051127-17273) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2}{add=[source_500 (1577281974672818176), source_501 (1577281974674915328), source_502 (1577281974674915329), source_503 (1577281974674915330), source_504 (1577281974674915331), source_505 (1577281974674915332), source_506 (1577281974674915333), source_507 (1577281974674915334), source_508 (1577281974674915335), source_509 (1577281974674915336), ... (100 adds)]} 0 3
[junit4] 2> 1727704 INFO (qtp6051127-17270) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577281974677012480,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1727704 INFO (qtp6051127-17270) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@179747f commitCommandVersion:1577281974677012480
[junit4] 2> 1727733 INFO (qtp6051127-17270) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@147950d[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 1727734 INFO (qtp6051127-17270) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1727735 INFO (searcherExecutor-8778-thread-1-processing-n:127.0.0.1:58255_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@147950d[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100) Uninverting(_3(7.0.0):C100) Uninverting(_4(7.0.0):C100) Uninverting(_5(7.0.0):C100)))}
[junit4] 2> 1727735 INFO (qtp6051127-17270) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 31
[junit4] 1> Adding 10 docs with commit=true, numDocs=700
[junit4] 2> 1727738 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2}{add=[source_600 (1577281974709518336), source_601 (1577281974711615488), source_602 (1577281974711615489), source_603 (1577281974711615490), source_604 (1577281974711615491), source_605 (1577281974711615492), source_606 (1577281974711615493), source_607 (1577281974711615494), source_608 (1577281974711615495), source_609 (1577281974711615496), ... (100 adds)]} 0 2
[junit4] 2> 1727739 INFO (qtp6051127-17268) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577281974713712640,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1727739 INFO (qtp6051127-17268) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@179747f commitCommandVersion:1577281974713712640
[junit4] 2> 1727771 INFO (qtp6051127-17268) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@10360e6[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 1727771 INFO (qtp6051127-17268) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1727773 INFO (searcherExecutor-8778-thread-1-processing-n:127.0.0.1:58255_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@10360e6[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100) Uninverting(_3(7.0.0):C100) Uninverting(_4(7.0.0):C100) Uninverting(_5(7.0.0):C100) Uninverting(_6(7.0.0):C100)))}
[junit4] 2> 1727773 INFO (qtp6051127-17268) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 33
[junit4] 1> Adding 10 docs with commit=true, numDocs=800
[junit4] 2> 1727777 INFO (qtp6051127-17271) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2}{add=[source_700 (1577281974750412800), source_701 (1577281974752509952), source_702 (1577281974752509953), source_703 (1577281974752509954), source_704 (1577281974752509955), source_705 (1577281974752509956), source_706 (1577281974752509957), source_707 (1577281974752509958), source_708 (1577281974752509959), source_709 (1577281974752509960), ... (100 adds)]} 0 3
[junit4] 2> 1727777 INFO (qtp6051127-17272) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577281974753558528,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1727777 INFO (qtp6051127-17272) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@179747f commitCommandVersion:1577281974753558528
[junit4] 2> 1727810 INFO (qtp6051127-17272) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1aad27d[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 1727811 INFO (qtp6051127-17272) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1727811 INFO (searcherExecutor-8778-thread-1-processing-n:127.0.0.1:58255_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@1aad27d[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100) Uninverting(_3(7.0.0):C100) Uninverting(_4(7.0.0):C100) Uninverting(_5(7.0.0):C100) Uninverting(_6(7.0.0):C100) Uninverting(_7(7.0.0):C100)))}
[junit4] 2> 1727813 INFO (qtp6051127-17272) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 34
[junit4] 1> Adding 10 docs with commit=true, numDocs=900
[junit4] 2> 1727816 INFO (qtp6051127-17267) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2}{add=[source_800 (1577281974791307264), source_801 (1577281974793404416), source_802 (1577281974793404417), source_803 (1577281974793404418), source_804 (1577281974793404419), source_805 (1577281974793404420), source_806 (1577281974793404421), source_807 (1577281974793404422), source_808 (1577281974793404423), source_809 (1577281974793404424), ... (100 adds)]} 0 3
[junit4] 2> 1727817 INFO (qtp6051127-17273) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577281974795501568,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1727817 INFO (qtp6051127-17273) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@179747f commitCommandVersion:1577281974795501568
[junit4] 2> 1727854 INFO (qtp6051127-17273) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@3f9c7f[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 1727855 INFO (qtp6051127-17273) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1727855 INFO (searcherExecutor-8778-thread-1-processing-n:127.0.0.1:58255_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@3f9c7f[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100) Uninverting(_3(7.0.0):C100) Uninverting(_4(7.0.0):C100) Uninverting(_5(7.0.0):C100) Uninverting(_6(7.0.0):C100) Uninverting(_7(7.0.0):C100) Uninverting(_8(7.0.0):C100)))}
[junit4] 2> 1727856 INFO (qtp6051127-17273) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 38
[junit4] 1> Adding 10 docs with commit=true, numDocs=1000
[junit4] 2> 1727860 INFO (qtp6051127-17270) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2}{add=[source_900 (1577281974837444608), source_901 (1577281974838493184), source_902 (1577281974838493185), source_903 (1577281974838493186), source_904 (1577281974838493187), source_905 (1577281974838493188), source_906 (1577281974838493189), source_907 (1577281974838493190), source_908 (1577281974838493191), source_909 (1577281974838493192), ... (100 adds)]} 0 3
[junit4] 2> 1727860 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577281974840590336,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 1727860 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@179747f commitCommandVersion:1577281974840590336
[junit4] 2> 1727905 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@14e9efb[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 1727905 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 1727906 INFO (searcherExecutor-8778-thread-1-processing-n:127.0.0.1:58255_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@14e9efb[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100) Uninverting(_3(7.0.0):C100) Uninverting(_4(7.0.0):C100) Uninverting(_5(7.0.0):C100) Uninverting(_6(7.0.0):C100) Uninverting(_7(7.0.0):C100) Uninverting(_8(7.0.0):C100) Uninverting(_9(7.0.0):C100)))}
[junit4] 2> 1727906 INFO (qtp6051127-17269) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 46
[junit4] 2> 1727907 INFO (qtp6051127-17268) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&_stateVer_=cdcr-source:4&wt=javabin&version=2} hits=1000 status=0 QTime=0
[junit4] 2> 1727907 INFO (qtp6051127-17268) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/get params={qt=/get&_stateVer_=cdcr-source:4&getVersions=1000&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 1727915 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e3a19892f0008, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1727917 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@1f2f92e{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 1727917 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=15604481
[junit4] 2> 1727917 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 1727917 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@16d06c9: rootName = solr_58255, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@cac7db
[junit4] 2> 1727921 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 1727921 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@1b66d40: rootName = solr_58255, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@11de6ce
[junit4] 2> 1727925 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 1727925 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@dc53ae: rootName = solr_58255, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@efd87f
[junit4] 2> 1727926 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 1727928 INFO (coreCloseExecutor-8783-thread-1) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@18c025f
[junit4] 2> 1727928 INFO (coreCloseExecutor-8783-thread-1) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.cdcr-source.shard1.replica_n1, tag=25952863
[junit4] 2> 1727928 INFO (coreCloseExecutor-8783-thread-1) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@121365: rootName = solr_58255, domain = solr.core.cdcr-source.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.cdcr-source.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@4a0017
[junit4] 2> 1727934 INFO (coreCloseExecutor-8783-thread-1) [n:127.0.0.1:58255_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.cdcr-source.shard1.leader, tag=25952863
[junit4] 2> 1727944 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.c.Overseer Overseer (id=98580123198881795-127.0.0.1:58255_solr-n_0000000000) closing
[junit4] 2> 1727944 INFO (OverseerStateUpdate-98580123198881795-127.0.0.1:58255_solr-n_0000000000) [n:127.0.0.1:58255_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:58255_solr
[junit4] 2> 1727946 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e3a19892f0003, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1727946 WARN (zkCallback-2643-thread-2-processing-n:127.0.0.1:58255_solr) [n:127.0.0.1:58255_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 1727946 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@198b5b0{/solr,null,UNAVAILABLE}
[junit4] 2> 1727946 INFO (zkCallback-2650-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
[junit4] 2> 1727947 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 1727948 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@1516b31{/solr,null,AVAILABLE}
[junit4] 2> 1727948 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.e.j.s.AbstractConnector Started ServerConnector@6d0853{HTTP/1.1,[http/1.1]}{127.0.0.1:58346}
[junit4] 2> 1727948 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.e.j.s.Server Started @1733575ms
[junit4] 2> 1727948 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=58346}
[junit4] 2> 1727948 ERROR (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 1727949 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr? version 7.0.0
[junit4] 2> 1727949 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 1727949 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 1727949 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-31T21:01:49.225Z
[junit4] 2> 1727953 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 1727961 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e3a19892f0009, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1727965 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58251/solr
[junit4] 2> 1727969 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e3a19892f000a, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1727977 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [n:127.0.0.1:58346_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 1727977 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [n:127.0.0.1:58346_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:58346_solr
[junit4] 2> 1727978 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [n:127.0.0.1:58346_solr ] o.a.s.c.Overseer Overseer (id=98580123198881803-127.0.0.1:58346_solr-n_0000000001) starting
[junit4] 2> 1727983 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [n:127.0.0.1:58346_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58346_solr
[junit4] 2> 1727986 INFO (zkCallback-2650-thread-1) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1727987 INFO (zkCallback-2656-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1728087 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [n:127.0.0.1:58346_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58346.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19390db
[junit4] 2> 1728093 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [n:127.0.0.1:58346_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58346.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19390db
[junit4] 2> 1728093 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [n:127.0.0.1:58346_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58346.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19390db
[junit4] 2> 1728096 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [n:127.0.0.1:58346_solr ] o.a.s.c.CorePropertiesLocator Found 1 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrBootstrapTest_5793E6163EF30DD-001\cdcr-source-001\node1\.
[junit4] 2> 1728096 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [n:127.0.0.1:58346_solr ] o.a.s.c.CorePropertiesLocator Cores are: [cdcr-source_shard1_replica_n1]
[junit4] 2> 1728098 INFO (coreLoadExecutor-8792-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1728201 INFO (zkCallback-2656-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1728202 INFO (zkCallback-2656-thread-2-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1729134 INFO (coreLoadExecutor-8792-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1729137 INFO (coreLoadExecutor-8792-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema.xml instead
[junit4] 2> 1729140 INFO (coreLoadExecutor-8792-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.IndexSchema [cdcr-source_shard1_replica_n1] Schema name=minimal
[junit4] 2> 1729143 INFO (coreLoadExecutor-8792-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1729144 INFO (coreLoadExecutor-8792-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.ManagedIndexSchema Created and persisted managed schema znode at /configs/cdcr-source/managed-schema
[junit4] 2> 1729146 INFO (coreLoadExecutor-8792-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, renamed the non-managed schema /configs/cdcr-source/schema.xml to /configs/cdcr-source/schema.xml.bak
[junit4] 2> 1729147 INFO (coreLoadExecutor-8792-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-source_shard1_replica_n1' using configuration from collection cdcr-source, trusted=true
[junit4] 2> 1729147 INFO (coreLoadExecutor-8792-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58346.solr.core.cdcr-source.shard1.replica_n1' (registry 'solr.core.cdcr-source.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19390db
[junit4] 2> 1729147 INFO (coreLoadExecutor-8792-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1729147 INFO (coreLoadExecutor-8792-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [[cdcr-source_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrBootstrapTest_5793E6163EF30DD-001\cdcr-source-001\node1\cdcr-source_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrBootstrapTest_5793E6163EF30DD-001\cdcr-source-001\node1\.\cdcr-source_shard1_replica_n1\data\]
[junit4] 2> 1729277 INFO (coreLoadExecutor-8792-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 1729277 INFO (coreLoadExecutor-8792-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 1729281 INFO (coreLoadExecutor-8792-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 1729281 INFO (coreLoadExecutor-8792-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 1729322 INFO (coreLoadExecutor-8792-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@17f52b1[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 1729323 INFO (coreLoadExecutor-8792-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-source
[junit4] 2> 1729324 INFO (coreLoadExecutor-8792-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-source
[junit4] 2> 1729324 INFO (coreLoadExecutor-8792-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-source/managed-schema
[junit4] 2> 1729324 INFO (coreLoadExecutor-8792-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 1729325 INFO (coreLoadExecutor-8792-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 1729331 INFO (coreLoadExecutor-8792-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrBufferStateManager Created znode /collections/cdcr-source/cdcr/state/buffer
[junit4] 2> 1729337 INFO (coreLoadExecutor-8792-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrProcessStateManager Created znode /collections/cdcr-source/cdcr/state/process
[junit4] 2> 1729369 WARN (cdcr-update-log-synchronizer-8798-thread-1) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrUpdateLogSynchronizer Caught unexpected exception
[junit4] 2> org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at http://127.0.0.1:58346/solr/cdcr-source_shard1_replica_n1: SolrCore is loading
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:627)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:253)
[junit4] 2> at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:242)
[junit4] 2> at org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1219)
[junit4] 2> at org.apache.solr.handler.CdcrUpdateLogSynchronizer$UpdateLogSynchronisation.run(CdcrUpdateLogSynchronizer.java:147)
[junit4] 2> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
[junit4] 2> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
[junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
[junit4] 2> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[junit4] 2> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1729373 INFO (searcherExecutor-8793-thread-1-processing-n:127.0.0.1:58346_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@17f52b1[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_a(7.0.0):C1000)))}
[junit4] 2> 1729379 INFO (coreZkRegister-8785-thread-1-processing-n:127.0.0.1:58346_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 1729379 INFO (coreZkRegister-8785-thread-1-processing-n:127.0.0.1:58346_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 1729379 INFO (coreZkRegister-8785-thread-1-processing-n:127.0.0.1:58346_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:58346/solr/cdcr-source_shard1_replica_n1/
[junit4] 2> 1729379 INFO (coreZkRegister-8785-thread-1-processing-n:127.0.0.1:58346_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 1729379 INFO (coreZkRegister-8785-thread-1-processing-n:127.0.0.1:58346_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:58346/solr/cdcr-source_shard1_replica_n1/ has no replicas
[junit4] 2> 1729381 INFO (coreZkRegister-8785-thread-1-processing-n:127.0.0.1:58346_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 1729381 INFO (zkCallback-2656-thread-2-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-source:shard1
[junit4] 2> 1729383 INFO (coreZkRegister-8785-thread-1-processing-n:127.0.0.1:58346_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:58346/solr/cdcr-source_shard1_replica_n1/ shard1
[junit4] 2> 1729385 INFO (coreZkRegister-8785-thread-1-processing-n:127.0.0.1:58346_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 1729490 INFO (zkCallback-2656-thread-2-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1729490 INFO (zkCallback-2656-thread-1-processing-n:127.0.0.1:58346_solr) [n:127.0.0.1:58346_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1729618 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.c.AbstractDistribZkTestBase Wait for recoveries to finish - collection: cdcr-source failOnTimeout:true timeout (sec):330
[junit4] 1> -
[junit4] 1> replica:core_node1 rstate:active live:true
[junit4] 1> no one is recoverying
[junit4] 2> 1729618 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.c.AbstractDistribZkTestBase Recoveries finished - collection: cdcr-source
[junit4] 2> 1729618 ERROR (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.c.s.i.CloudSolrClient Request to collection cdcr-source failed due to (510) org.apache.solr.common.SolrException: Could not find a healthy node to handle the request., retry? 0
[junit4] 2> 1729619 WARN (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.c.s.i.CloudSolrClient Re-trying request to collection(s) cdcr-source after stale state error from server.
[junit4] 2> 1729621 INFO (qtp11649075-17326) [n:127.0.0.1:58346_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&_stateVer_=cdcr-source:7&wt=javabin&version=2} hits=1000 status=0 QTime=0
[junit4] 2> 1729643 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e3a1987c20006, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 1729646 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 1729647 INFO (TEST-CdcrBootstrapTest.testConvertClusterToCdcrAndBootstrap-seed#[5793E6163EF30DD]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:58227/solr ready
[junit4] 2> 1729649 INFO (qtp22723055-17227) [n:127.0.0.1:58231_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=cdcr-target&name=cdcr-target&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 1729651 INFO (OverseerThreadFactory-8767-thread-1-processing-n:127.0.0.1:58231_solr) [n:127.0.0.1:58231_solr ] o.a.s.c.CreateCollectionCmd Create collection cdcr-target
[junit4] 2> 1729761 INFO (OverseerStateUpdate-98580123174961155-127.0.0.1:58231_solr-n_0000000000) [n:127.0.0.1:58231_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"cdcr-target",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"cdcr-target_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:58231/solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 1729963 INFO (qtp22723055-17228) [n:127.0.0.1:58231_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node1&collection.configName=cdcr-target&newCollection=true&name=cdcr-target_shard1_replica_n1&action=CREATE&numShards=1&collection=cdcr-target&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 1729964 INFO (qtp22723055-17228) [n:127.0.0.1:58231_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 1730070 INFO (zkCallback-2631-thread-1-processing-n:127.0.0.1:58231_solr) [n:127.0.0.1:58231_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1730071 INFO (zkCallback-2631-thread-2-processing-n:127.0.0.1:58231_solr) [n:127.0.0.1:58231_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 1730999 INFO (qtp22723055-17228) [n:127.0.0.1:58231_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 1731004 INFO (qtp22723055-17228) [n:127.0.0.1:58231_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema.xml instead
[junit4] 2> 1731010 INFO (qtp22723055-17228) [n:127.0.0.1:58231_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.s.IndexSchema [cdcr-target_shard1_replica_n1] Schema name=minimal
[junit4] 2> 1731017 INFO (qtp22723055-17228) [n:127.0.0.1:58231_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 1731018 INFO (qtp22723055-17228) [n:127.0.0.1:58231_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.s.ManagedIndexSchema Created and persisted managed schema znode at /configs/cdcr-target/managed-schema
[junit4] 2> 1731023 INFO (qtp22723055-17228) [n:127.0.0.1:58231_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, renamed the non-managed schema /configs/cdcr-target/schema.xml to /configs/cdcr-target/schema.xml.bak
[junit4] 2> 1731023 INFO (qtp22723055-17228) [n:127.0.0.1:58231_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-target_shard1_replica_n1' using configuration from collection cdcr-target, trusted=true
[junit4] 2> 1731023 INFO (qtp22723055-17228) [n:127.0.0.1:58231_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58231.solr.core.cdcr-target.shard1.replica_n1' (registry 'solr.core.cdcr-target.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@19390db
[junit4] 2> 1731024 INFO (qtp22723055-17228) [n:127.0.0.1:58231_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 1731024 INFO (qtp22723055-17228) [n:127.0.0.1:58231_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.SolrCore [[cdcr
[...truncated too long message...]
strapWithContinousIndexingOnSourceCluster-seed#[5793E6163EF30DD]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 1879754 INFO (TEST-CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster-seed#[5793E6163EF30DD]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:58591 58591
[junit4] 2> 1879755 INFO (Thread-3898) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:58591 58591
[junit4] 2> 1879756 WARN (Thread-3898) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2> 2 /solr/configs/cdcr-target/managed-schema
[junit4] 2> 2 /solr/collections/cdcr-target/leaders/shard1/leader
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/collections/cdcr-target/state.json
[junit4] 2> 3 /solr/collections/cdcr-target/cdcr/state/process
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 2 /solr/collections/cdcr-target/cdcr/state/buffer
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/live_nodes
[junit4] 2> 3 /solr/collections
[junit4] 2>
[junit4] 2> 1879756 INFO (TEST-CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster-seed#[5793E6163EF30DD]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testBootstrapWithContinousIndexingOnSourceCluster
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CdcrBootstrapTest -Dtests.method=testBootstrapWithContinousIndexingOnSourceCluster -Dtests.seed=5793E6163EF30DD -Dtests.slow=true -Dtests.locale=ja -Dtests.timezone=America/Swift_Current -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
[junit4] FAILURE 131s J1 | CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster <<<
[junit4] > Throwable #1: java.lang.AssertionError: Document mismatch on target after sync expected:<2000> but was:<1100>
[junit4] > at __randomizedtesting.SeedInfo.seed([5793E6163EF30DD:D13C753884B98326]:0)
[junit4] > at org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster(CdcrBootstrapTest.java:309)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: leaving temporary files on disk at: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J1\temp\solr.cloud.CdcrBootstrapTest_5793E6163EF30DD-001
[junit4] 2> Aug 31, 2017 9:04:21 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 6 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=Asserting(Lucene70): {_version_=PostingsFormat(name=MockRandom), xyz=FST50, id=TestBloomFilteredLucenePostings(BloomFilteringPostingsFormat(Lucene50(blocksize=128)))}, docValues:{_version_=DocValuesFormat(name=Asserting), xyz=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=1108, maxMBSortInHeap=6.825232481550216, sim=RandomSimilarity(queryNorm=false): {}, locale=ja, timezone=America/Swift_Current
[junit4] 2> NOTE: Windows 10 10.0 x86/Oracle Corporation 1.8.0_144 (32-bit)/cpus=3,threads=1,free=384359376,total=536608768
[junit4] 2> NOTE: All tests run in this JVM: [TestFieldCacheWithThreads, TestRestManager, TestDelegationWithHadoopAuth, OpenExchangeRatesOrgProviderTest, DOMUtilTest, DocValuesMultiTest, CdcrReplicationDistributedZkTest, TestJsonRequest, StatsComponentTest, TestInitQParser, TestDFISimilarityFactory, TestManagedResource, TestSortByMinMaxFunction, SolrCoreCheckLockOnStartupTest, TestRecovery, FileUtilsTest, TestFileDictionaryLookup, TestUnifiedSolrHighlighter, TestAuthorizationFramework, TestSolrIndexConfig, BaseCdcrDistributedZkTest, TestLMJelinekMercerSimilarityFactory, SortSpecParsingTest, TestSolrCLIRunExample, TestSchemaSimilarityResource, TestBadConfig, TemplateUpdateProcessorTest, TestNumericRangeQuery32, SchemaVersionSpecificBehaviorTest, TestBM25SimilarityFactory, JSONWriterTest, TestComplexPhraseQParserPlugin, LoggingHandlerTest, DistributedFacetPivotLargeTest, BitVectorTest, TestPostingsSolrHighlighter, TestTlogReplica, MergeStrategyTest, TestDistributedGrouping, EchoParamsTest, HLLUtilTest, SearchHandlerTest, DeleteLastCustomShardedReplicaTest, LeaderInitiatedRecoveryOnCommitTest, SynonymTokenizerTest, TestDefaultStatsCache, JvmMetricsTest, DistributedExpandComponentTest, TestSolrJ, TestGraphTermsQParserPlugin, CurrencyFieldTypeTest, TestChildDocTransformer, TestSystemIdResolver, OpenCloseCoreStressTest, OverriddenZkACLAndCredentialsProvidersTest, PreAnalyzedUpdateProcessorTest, TestExactSharedStatsCache, TestSolrFieldCacheBean, TestRuleBasedAuthorizationPlugin, UpdateRequestProcessorFactoryTest, TestDistributedMissingSort, TestRestoreCore, ConvertedLegacyTest, TestClassNameShortening, HdfsCollectionsAPIDistributedZkTest, HdfsBasicDistributedZk2Test, TestSchemaManager, TestWordDelimiterFilterFactory, TestBackupRepositoryFactory, DistanceUnitsTest, TestJoin, TestSolrCoreSnapshots, TestManagedSchema, BasicFunctionalityTest, TestConfigSetsAPIExclusivity, PingRequestHandlerTest, TestRandomCollapseQParserPlugin, TestJsonFacetRefinement, TestReloadDeadlock, OverseerStatusTest, CheckHdfsIndexTest, TestOnReconnectListenerSupport, BadComponentTest, CollectionsAPIAsyncDistributedZkTest, TestRandomFlRTGCloud, TestSSLRandomization, TestSQLHandlerNonCloud, TestRawTransformer, TestMergePolicyConfig, AnalysisErrorHandlingTest, AnalyticsMergeStrategyTest, StatsReloadRaceTest, TestDynamicFieldCollectionResource, DeleteReplicaTest, SolrCloudExampleTest, TestLRUStatsCache, TestLegacyTerms, DistributedFacetPivotWhiteBoxTest, TestConfigReload, TestFastLRUCache, CloneFieldUpdateProcessorFactoryTest, TestPullReplica, SolrJmxReporterCloudTest, ShowFileRequestHandlerTest, TestUninvertingReader, MoreLikeThisHandlerTest, FullHLLTest, BigEndianAscendingWordDeserializerTest, TestCloudNestedDocsSort, SignatureUpdateProcessorFactoryTest, TestSolr4Spatial, TestManagedSchemaThreadSafety, DistributedIntervalFacetingTest, RulesTest, SolrCmdDistributorTest, MetricsHandlerTest, SpellCheckCollatorTest, TestZkChroot, DistributedMLTComponentTest, TestStressInPlaceUpdates, TestFiltering, TestConfigSetImmutable, TestSolrQueryParser, CoreAdminCreateDiscoverTest, SolrPluginUtilsTest, PluginInfoTest, ReplicationFactorTest, LukeRequestHandlerTest, SolrGangliaReporterTest, TestRangeQuery, TestManagedStopFilterFactory, TestXmlQParser, TestFastWriter, TestCloudPivotFacet, TolerantUpdateProcessorTest, CdcrRequestHandlerTest, AutoScalingHandlerTest, BasicAuthStandaloneTest, PeerSyncReplicationTest, TestRealTimeGet, TestPerFieldSimilarityWithDefaultOverride, HdfsDirectoryFactoryTest, AutoCommitTest, TestCharFilters, ChaosMonkeyNothingIsSafeWithPullReplicasTest, TestStressUserVersions, HdfsThreadLeakTest, ImplicitSnitchTest, TermsComponentTest, TestAuthenticationFramework, BasicDistributedZk2Test, HttpSolrCallGetCoreTest, TestFieldSortValues, TestQueryUtils, TestMultiWordSynonyms, TestRebalanceLeaders, PolyFieldTest, TestSearchPerf, ConfigureRecoveryStrategyTest, SolrIndexConfigTest, JavabinLoaderTest, ZkSolrClientTest, TestHighFrequencyDictionaryFactory, TestAnalyzedSuggestions, TestSolrCloudWithSecureImpersonation, ReplicaListTransformerTest, TestSolrXml, DirectoryFactoryTest, SimpleMLTQParserTest, DirectSolrSpellCheckerTest, TestFieldTypeCollectionResource, HdfsUnloadDistributedZkTest, DocumentBuilderTest, SharedFSAutoReplicaFailoverTest, TestCoreAdminApis, RollingRestartTest, TestTolerantUpdateProcessorRandomCloud, TestRTGBase, ChaosMonkeySafeLeaderTest, UnloadDistributedZkTest, CollectionsAPIDistributedZkTest, SyncSliceTest, LeaderElectionIntegrationTest, BasicZkTest, RecoveryZkTest, FullSolrCloudDistribCmdsTest, TestRandomFaceting, LeaderElectionTest, TestRandomDVFaceting, ZkCLITest, ShardRoutingCustomTest, TestFaceting, TestHashPartitioner, TermVectorComponentDistributedTest, ZkControllerTest, TestReload, TestStressVersions, DistributedTermsComponentTest, SimpleFacetsTest, SolrCoreTest, SpellCheckComponentTest, TestGroupingSearch, QueryElevationComponentTest, PeerSyncTest, TestFunctionQuery, DirectUpdateHandlerTest, HighlighterTest, SoftAutoCommitTest, SolrIndexSplitterTest, SimplePostToolTest, TestCoreDiscovery, TestExtendedDismaxParser, SolrRequestParserTest, SuggesterTSTTest, TestStressLucene, TestTrie, WordBreakSolrSpellCheckerTest, FieldMutatingUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, TestSolrDeletionPolicy1, SolrInfoBeanTest, CacheHeaderTest, IndexBasedSpellCheckerTest, DocumentAnalysisRequestHandlerTest, TestOmitPositions, XmlUpdateRequestHandlerTest, PathHierarchyTokenizerFactoryTest, RequiredFieldsTest, IndexSchemaRuntimeFieldTest, TestJmxIntegration, TestCSVResponseWriter, QueryParsingTest, UniqFieldsUpdateProcessorFactoryTest, TestPartialUpdateDeduplication, TestComponentsName, HighlighterConfigTest, SOLR749Test, UpdateParamsTest, TestQuerySenderNoQuery, CopyFieldTest, ResponseLogComponentTest, TestSolrDeletionPolicy2, MultiTermTest, SampleTest, TestDocSet, TestElisionMultitermQuery, TestConfig, ExternalFileFieldSortTest, TestSolrCoreProperties, NotRequiredUniqueKeyTest, TestPhraseSuggestions, TestSweetSpotSimilarityFactory, ResourceLoaderTest, DateMathParserTest, TestSuggestSpellingConverter, ClusterStateTest, TestDocumentBuilder, SliceStateTest, SystemInfoHandlerTest, CursorPagingTest, TestEmbeddedSolrServerConstructors, TestEmbeddedSolrServerSchemaAPI, TestJettySolrRunner, ActionThrottleTest, AssignTest, CdcrBootstrapTest]
[junit4] Completed [517/729 (1!)] on J1 in 156.78s, 3 tests, 1 failure <<< FAILURES!
[...truncated 40065 lines...]
[JENKINS] Lucene-Solr-7.0-Windows (32bit/jdk1.8.0_144) - Build #
123 - Still Unstable!
Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.0-Windows/123/
Java: 32bit/jdk1.8.0_144 -client -XX:+UseConcMarkSweepGC
1 tests failed.
FAILED: org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster
Error Message:
Document mismatch on target after sync expected:<2000> but was:<1100>
Stack Trace:
java.lang.AssertionError: Document mismatch on target after sync expected:<2000> but was:<1100>
at __randomizedtesting.SeedInfo.seed([243EB6F346B65BA1:F07BFDAAA1E0E85A]:0)
at org.junit.Assert.fail(Assert.java:93)
at org.junit.Assert.failNotEquals(Assert.java:647)
at org.junit.Assert.assertEquals(Assert.java:128)
at org.junit.Assert.assertEquals(Assert.java:472)
at org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster(CdcrBootstrapTest.java:309)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1713)
at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:907)
at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:943)
at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:957)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:916)
at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:802)
at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:852)
at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
at java.lang.Thread.run(Thread.java:748)
Build Log:
[...truncated 10890 lines...]
[junit4] Suite: org.apache.solr.cloud.CdcrBootstrapTest
[junit4] 2> Creating dataDir: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrBootstrapTest_243EB6F346B65BA1-001\init-core-data-001
[junit4] 2> 22648 INFO (SUITE-CdcrBootstrapTest-seed#[243EB6F346B65BA1]-worker) [ ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=false
[junit4] 2> 22653 INFO (SUITE-CdcrBootstrapTest-seed#[243EB6F346B65BA1]-worker) [ ] o.a.s.SolrTestCaseJ4 Randomized ssl (false) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason=, value=NaN, ssl=NaN, clientAuth=NaN)
[junit4] 2> 22655 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[243EB6F346B65BA1]) [ ] o.a.s.SolrTestCaseJ4 ###Starting testBootstrapWithSourceCluster
[junit4] 2> 22668 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[243EB6F346B65BA1]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrBootstrapTest_243EB6F346B65BA1-001\cdcr-target-001
[junit4] 2> 22670 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[243EB6F346B65BA1]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 22674 INFO (Thread-8) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 22674 INFO (Thread-8) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 22680 ERROR (Thread-8) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 22784 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[243EB6F346B65BA1]) [ ] o.a.s.c.ZkTestServer start zk server on port:58781
[junit4] 2> 24068 INFO (jetty-launcher-25-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 24232 INFO (jetty-launcher-25-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@d21afa{/solr,null,AVAILABLE}
[junit4] 2> 24271 INFO (jetty-launcher-25-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@1048f9{HTTP/1.1,[http/1.1]}{127.0.0.1:58785}
[junit4] 2> 24271 INFO (jetty-launcher-25-thread-1) [ ] o.e.j.s.Server Started @28092ms
[junit4] 2> 24271 INFO (jetty-launcher-25-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=58785}
[junit4] 2> 24360 ERROR (jetty-launcher-25-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 24360 INFO (jetty-launcher-25-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 24360 INFO (jetty-launcher-25-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 24360 INFO (jetty-launcher-25-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 24361 INFO (jetty-launcher-25-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-31T09:21:17.480Z
[junit4] 2> 24377 INFO (jetty-launcher-25-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 24480 INFO (jetty-launcher-25-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58781/solr
[junit4] 2> 24484 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e379836170002, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 24741 INFO (jetty-launcher-25-thread-1) [n:127.0.0.1:58785_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 24741 INFO (jetty-launcher-25-thread-1) [n:127.0.0.1:58785_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:58785_solr
[junit4] 2> 24744 INFO (jetty-launcher-25-thread-1) [n:127.0.0.1:58785_solr ] o.a.s.c.Overseer Overseer (id=98577368730763267-127.0.0.1:58785_solr-n_0000000000) starting
[junit4] 2> 24756 INFO (jetty-launcher-25-thread-1) [n:127.0.0.1:58785_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58785_solr
[junit4] 2> 24761 INFO (zkCallback-32-thread-1-processing-n:127.0.0.1:58785_solr) [n:127.0.0.1:58785_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 25129 INFO (jetty-launcher-25-thread-1) [n:127.0.0.1:58785_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58785.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@18ab29c
[junit4] 2> 25130 INFO (jetty-launcher-25-thread-1) [n:127.0.0.1:58785_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58785.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@18ab29c
[junit4] 2> 25130 INFO (jetty-launcher-25-thread-1) [n:127.0.0.1:58785_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58785.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@18ab29c
[junit4] 2> 25139 INFO (jetty-launcher-25-thread-1) [n:127.0.0.1:58785_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrBootstrapTest_243EB6F346B65BA1-001\cdcr-target-001\node1\.
[junit4] 1> Target zkHost = 127.0.0.1:58781/solr
[junit4] 2> 25508 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[243EB6F346B65BA1]) [ ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 1 servers in C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrBootstrapTest_243EB6F346B65BA1-001\cdcr-source-001
[junit4] 2> 25508 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[243EB6F346B65BA1]) [ ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
[junit4] 2> 25509 INFO (Thread-19) [ ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
[junit4] 2> 25509 INFO (Thread-19) [ ] o.a.s.c.ZkTestServer Starting server
[junit4] 2> 25520 ERROR (Thread-19) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 25610 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[243EB6F346B65BA1]) [ ] o.a.s.c.ZkTestServer start zk server on port:58809
[junit4] 2> 25619 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e3798412e0000, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 25621 INFO (jetty-launcher-37-thread-1) [ ] o.e.j.s.Server jetty-9.3.14.v20161028
[junit4] 2> 25622 INFO (jetty-launcher-37-thread-1) [ ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@d97279{/solr,null,AVAILABLE}
[junit4] 2> 25625 INFO (jetty-launcher-37-thread-1) [ ] o.e.j.s.AbstractConnector Started ServerConnector@14579d9{HTTP/1.1,[http/1.1]}{127.0.0.1:58814}
[junit4] 2> 25625 INFO (jetty-launcher-37-thread-1) [ ] o.e.j.s.Server Started @29446ms
[junit4] 2> 25625 INFO (jetty-launcher-37-thread-1) [ ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=58814}
[junit4] 2> 25625 ERROR (jetty-launcher-37-thread-1) [ ] o.a.s.s.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
[junit4] 2> 25626 INFO (jetty-launcher-37-thread-1) [ ] o.a.s.s.SolrDispatchFilter ___ _ Welcome to Apache Solr™ version 7.0.0
[junit4] 2> 25627 INFO (jetty-launcher-37-thread-1) [ ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _ Starting in cloud mode on port null
[junit4] 2> 25627 INFO (jetty-launcher-37-thread-1) [ ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_| Install dir: null, Default config dir: null
[junit4] 2> 25627 INFO (jetty-launcher-37-thread-1) [ ] o.a.s.s.SolrDispatchFilter |___/\___/_|_| Start time: 2017-08-31T09:21:18.746Z
[junit4] 2> 25632 INFO (jetty-launcher-37-thread-1) [ ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
[junit4] 2> 25659 INFO (jetty-launcher-37-thread-1) [ ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:58809/solr
[junit4] 2> 25665 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e3798412e0002, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 25847 INFO (jetty-launcher-37-thread-1) [n:127.0.0.1:58814_solr ] o.a.s.c.Overseer Overseer (id=null) closing
[junit4] 2> 25849 INFO (jetty-launcher-37-thread-1) [n:127.0.0.1:58814_solr ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:58814_solr
[junit4] 2> 25851 INFO (jetty-launcher-37-thread-1) [n:127.0.0.1:58814_solr ] o.a.s.c.Overseer Overseer (id=98577368916819971-127.0.0.1:58814_solr-n_0000000000) starting
[junit4] 2> 25864 INFO (jetty-launcher-37-thread-1) [n:127.0.0.1:58814_solr ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:58814_solr
[junit4] 2> 25875 INFO (OverseerStateUpdate-98577368916819971-127.0.0.1:58814_solr-n_0000000000) [n:127.0.0.1:58814_solr ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 26312 INFO (jetty-launcher-37-thread-1) [n:127.0.0.1:58814_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58814.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@18ab29c
[junit4] 2> 26335 INFO (jetty-launcher-37-thread-1) [n:127.0.0.1:58814_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58814.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@18ab29c
[junit4] 2> 26336 INFO (jetty-launcher-37-thread-1) [n:127.0.0.1:58814_solr ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58814.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@18ab29c
[junit4] 2> 26338 INFO (jetty-launcher-37-thread-1) [n:127.0.0.1:58814_solr ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrBootstrapTest_243EB6F346B65BA1-001\cdcr-source-001\node1\.
[junit4] 2> 26442 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[243EB6F346B65BA1]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 26444 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[243EB6F346B65BA1]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:58809/solr ready
[junit4] 2> 27113 INFO (qtp3048805-163) [n:127.0.0.1:58814_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=cdcr-source&name=cdcr-source&nrtReplicas=1&action=CREATE&numShards=1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 27152 INFO (OverseerThreadFactory-73-thread-1-processing-n:127.0.0.1:58814_solr) [n:127.0.0.1:58814_solr ] o.a.s.c.CreateCollectionCmd Create collection cdcr-source
[junit4] 2> 27283 INFO (OverseerStateUpdate-98577368916819971-127.0.0.1:58814_solr-n_0000000000) [n:127.0.0.1:58814_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"cdcr-source",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"cdcr-source_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:58814/solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 27514 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&collection.configName=cdcr-source&newCollection=true&collection=cdcr-source&version=2&replicaType=NRT&coreNodeName=core_node1&name=cdcr-source_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin
[junit4] 2> 27522 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 27661 INFO (zkCallback-44-thread-1-processing-n:127.0.0.1:58814_solr) [n:127.0.0.1:58814_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 27665 INFO (zkCallback-44-thread-2-processing-n:127.0.0.1:58814_solr) [n:127.0.0.1:58814_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 28613 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 28658 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema.xml instead
[junit4] 2> 28667 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.IndexSchema [cdcr-source_shard1_replica_n1] Schema name=minimal
[junit4] 2> 28676 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 28677 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.ManagedIndexSchema Created and persisted managed schema znode at /configs/cdcr-source/managed-schema
[junit4] 2> 28683 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, renamed the non-managed schema /configs/cdcr-source/schema.xml to /configs/cdcr-source/schema.xml.bak
[junit4] 2> 28685 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-source_shard1_replica_n1' using configuration from collection cdcr-source, trusted=true
[junit4] 2> 28686 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58814.solr.core.cdcr-source.shard1.replica_n1' (registry 'solr.core.cdcr-source.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@18ab29c
[junit4] 2> 28691 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 28691 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [[cdcr-source_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrBootstrapTest_243EB6F346B65BA1-001\cdcr-source-001\node1\cdcr-source_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrBootstrapTest_243EB6F346B65BA1-001\cdcr-source-001\node1\.\cdcr-source_shard1_replica_n1\data\]
[junit4] 2> 29013 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 29013 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 29016 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 29016 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 29022 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@26c0c5[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 29029 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-source
[junit4] 2> 29030 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-source
[junit4] 2> 29040 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-source/managed-schema
[junit4] 2> 29046 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 29047 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 29080 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrBufferStateManager Created znode /collections/cdcr-source/cdcr/state/buffer
[junit4] 2> 29096 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrProcessStateManager Created znode /collections/cdcr-source/cdcr/state/process
[junit4] 2> 29152 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577237906478596096
[junit4] 2> 29167 INFO (searcherExecutor-76-thread-1-processing-n:127.0.0.1:58814_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@26c0c5[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 29194 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 29195 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 29199 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:58814/solr/cdcr-source_shard1_replica_n1/
[junit4] 2> 29205 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 29205 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:58814/solr/cdcr-source_shard1_replica_n1/ has no replicas
[junit4] 2> 29207 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 29271 INFO (zkCallback-44-thread-2-processing-n:127.0.0.1:58814_solr) [n:127.0.0.1:58814_solr ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-source:shard1
[junit4] 2> 29279 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:58814/solr/cdcr-source_shard1_replica_n1/ shard1
[junit4] 2> 29385 INFO (zkCallback-44-thread-2-processing-n:127.0.0.1:58814_solr) [n:127.0.0.1:58814_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 29385 INFO (zkCallback-44-thread-1-processing-n:127.0.0.1:58814_solr) [n:127.0.0.1:58814_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 29431 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 29433 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&collection.configName=cdcr-source&newCollection=true&collection=cdcr-source&version=2&replicaType=NRT&coreNodeName=core_node1&name=cdcr-source_shard1_replica_n1&action=CREATE&numShards=1&shard=shard1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin} status=0 QTime=1924
[junit4] 2> 29538 INFO (zkCallback-44-thread-1-processing-n:127.0.0.1:58814_solr) [n:127.0.0.1:58814_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 29538 INFO (zkCallback-44-thread-2-processing-n:127.0.0.1:58814_solr) [n:127.0.0.1:58814_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/state.json] for collection [cdcr-source] has occurred - updating... (live nodes size: [1])
[junit4] 2> 29584 INFO (qtp3048805-163) [n:127.0.0.1:58814_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 29584 INFO (qtp3048805-163) [n:127.0.0.1:58814_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=cdcr-source&name=cdcr-source&nrtReplicas=1&action=CREATE&numShards=1&property.solr.directoryFactory=solr.StandardDirectoryFactory&wt=javabin&version=2} status=0 QTime=2483
[junit4] 1> Adding 10 docs with commit=true, numDocs=100
[junit4] 2> 29757 INFO (qtp3048805-168) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=124
[junit4] 2> 29764 INFO (qtp3048805-167) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577237907120324608,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 29764 INFO (qtp3048805-167) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1779ed5 commitCommandVersion:1577237907120324608
[junit4] 2> 29802 INFO (qtp3048805-167) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@dbbd79[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 29803 INFO (qtp3048805-167) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 29808 INFO (searcherExecutor-76-thread-1-processing-n:127.0.0.1:58814_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@dbbd79[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100)))}
[junit4] 2> 29808 INFO (qtp3048805-167) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=49
[junit4] 1> Adding 10 docs with commit=true, numDocs=200
[junit4] 2> 29846 INFO (qtp3048805-166) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=34
[junit4] 2> 29849 INFO (qtp3048805-161) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577237907209453568,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 29849 INFO (qtp3048805-161) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1779ed5 commitCommandVersion:1577237907209453568
[junit4] 2> 29878 INFO (qtp3048805-161) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@16f891e[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 29880 INFO (qtp3048805-161) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 29886 INFO (searcherExecutor-76-thread-1-processing-n:127.0.0.1:58814_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@16f891e[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100)))}
[junit4] 2> 29887 INFO (qtp3048805-161) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=38
[junit4] 1> Adding 10 docs with commit=true, numDocs=300
[junit4] 2> 29911 INFO (qtp3048805-164) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=22
[junit4] 2> 29913 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577237907276562432,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 29914 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1779ed5 commitCommandVersion:1577237907276562432
[junit4] 2> 29941 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@163e631[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 29942 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 29944 INFO (searcherExecutor-76-thread-1-processing-n:127.0.0.1:58814_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@163e631[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100)))}
[junit4] 2> 29944 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=31
[junit4] 1> Adding 10 docs with commit=true, numDocs=400
[junit4] 2> 29972 INFO (qtp3048805-163) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=23
[junit4] 2> 29974 INFO (qtp3048805-168) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577237907340525568,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 29974 INFO (qtp3048805-168) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1779ed5 commitCommandVersion:1577237907340525568
[junit4] 2> 30010 INFO (qtp3048805-168) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1bb1dd[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 30011 INFO (qtp3048805-168) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 30014 INFO (searcherExecutor-76-thread-1-processing-n:127.0.0.1:58814_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@1bb1dd[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100) Uninverting(_3(7.0.0):C100)))}
[junit4] 2> 30014 INFO (qtp3048805-168) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=41
[junit4] 1> Adding 10 docs with commit=true, numDocs=500
[junit4] 2> 30039 INFO (qtp3048805-167) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=22
[junit4] 2> 30043 INFO (qtp3048805-166) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577237907412877312,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 30043 INFO (qtp3048805-166) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1779ed5 commitCommandVersion:1577237907412877312
[junit4] 2> 30128 INFO (qtp3048805-166) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@10c23f5[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 30131 INFO (qtp3048805-166) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 30134 INFO (searcherExecutor-76-thread-1-processing-n:127.0.0.1:58814_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@10c23f5[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100) Uninverting(_3(7.0.0):C100) Uninverting(_4(7.0.0):C100)))}
[junit4] 2> 30134 INFO (qtp3048805-166) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=93
[junit4] 1> Adding 10 docs with commit=true, numDocs=600
[junit4] 2> 30156 INFO (qtp3048805-161) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=18
[junit4] 2> 30159 INFO (qtp3048805-164) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577237907534512128,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 30159 INFO (qtp3048805-164) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1779ed5 commitCommandVersion:1577237907534512128
[junit4] 2> 30206 INFO (qtp3048805-164) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1f5195b[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 30207 INFO (qtp3048805-164) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 30210 INFO (searcherExecutor-76-thread-1-processing-n:127.0.0.1:58814_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@1f5195b[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100) Uninverting(_3(7.0.0):C100) Uninverting(_4(7.0.0):C100) Uninverting(_5(7.0.0):C100)))}
[junit4] 2> 30211 INFO (qtp3048805-164) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=52
[junit4] 1> Adding 10 docs with commit=true, numDocs=700
[junit4] 2> 30232 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=19
[junit4] 2> 30234 INFO (qtp3048805-163) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577237907613155328,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 30234 INFO (qtp3048805-163) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1779ed5 commitCommandVersion:1577237907613155328
[junit4] 2> 30274 INFO (qtp3048805-163) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@182e6c3[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 30275 INFO (qtp3048805-163) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 30281 INFO (searcherExecutor-76-thread-1-processing-n:127.0.0.1:58814_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@182e6c3[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100) Uninverting(_3(7.0.0):C100) Uninverting(_4(7.0.0):C100) Uninverting(_5(7.0.0):C100) Uninverting(_6(7.0.0):C100)))}
[junit4] 2> 30282 INFO (qtp3048805-163) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=47
[junit4] 1> Adding 10 docs with commit=true, numDocs=800
[junit4] 2> 30305 INFO (qtp3048805-168) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=19
[junit4] 2> 30307 INFO (qtp3048805-167) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577237907689701376,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 30307 INFO (qtp3048805-167) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1779ed5 commitCommandVersion:1577237907689701376
[junit4] 2> 30361 INFO (qtp3048805-167) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@1dab212[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 30362 INFO (qtp3048805-167) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 30368 INFO (searcherExecutor-76-thread-1-processing-n:127.0.0.1:58814_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@1dab212[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100) Uninverting(_3(7.0.0):C100) Uninverting(_4(7.0.0):C100) Uninverting(_5(7.0.0):C100) Uninverting(_6(7.0.0):C100) Uninverting(_7(7.0.0):C100)))}
[junit4] 2> 30368 INFO (qtp3048805-167) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=62
[junit4] 1> Adding 10 docs with commit=true, numDocs=900
[junit4] 2> 30386 INFO (qtp3048805-166) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=16
[junit4] 2> 30388 INFO (qtp3048805-161) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577237907774636032,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 30388 INFO (qtp3048805-161) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1779ed5 commitCommandVersion:1577237907774636032
[junit4] 2> 30453 INFO (qtp3048805-161) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@163d637[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 30454 INFO (qtp3048805-161) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 30461 INFO (searcherExecutor-76-thread-1-processing-n:127.0.0.1:58814_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@163d637[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100) Uninverting(_3(7.0.0):C100) Uninverting(_4(7.0.0):C100) Uninverting(_5(7.0.0):C100) Uninverting(_6(7.0.0):C100) Uninverting(_7(7.0.0):C100) Uninverting(_8(7.0.0):C100)))}
[junit4] 2> 30461 INFO (qtp3048805-161) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=73
[junit4] 1> Adding 10 docs with commit=true, numDocs=1000
[junit4] 2> 30479 INFO (qtp3048805-164) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&wt=javabin&version=2} status=0 QTime=15
[junit4] 2> 30481 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577237907872153600,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 30481 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1779ed5 commitCommandVersion:1577237907872153600
[junit4] 2> 30533 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@148ce9e[cdcr-source_shard1_replica_n1] main]
[junit4] 2> 30534 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 30539 INFO (searcherExecutor-76-thread-1-processing-n:127.0.0.1:58814_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-source_shard1_replica_n1] Registered new searcher Searcher@148ce9e[cdcr-source_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.0.0):C100) Uninverting(_1(7.0.0):C100) Uninverting(_2(7.0.0):C100) Uninverting(_3(7.0.0):C100) Uninverting(_4(7.0.0):C100) Uninverting(_5(7.0.0):C100) Uninverting(_6(7.0.0):C100) Uninverting(_7(7.0.0):C100) Uninverting(_8(7.0.0):C100) Uninverting(_9(7.0.0):C100)))}
[junit4] 2> 30540 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-source:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=59
[junit4] 2> 30607 INFO (qtp3048805-163) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&_stateVer_=cdcr-source:4&wt=javabin&version=2} hits=1000 status=0 QTime=50
[junit4] 2> 30649 WARN (NIOServerCxn.Factory:0.0.0.0/0.0.0.0:0) [ ] o.a.z.s.NIOServerCnxn caught end of stream exception
[junit4] 2> EndOfStreamException: Unable to read additional data from client sessionid 0x15e379836170006, likely client has closed socket
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:239)
[junit4] 2> at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:203)
[junit4] 2> at java.lang.Thread.run(Thread.java:748)
[junit4] 2> 30664 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[243EB6F346B65BA1]) [ ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 30668 INFO (TEST-CdcrBootstrapTest.testBootstrapWithSourceCluster-seed#[243EB6F346B65BA1]) [ ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:58781/solr ready
[junit4] 2> 30675 INFO (qtp13866140-120) [n:127.0.0.1:58785_solr ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params replicationFactor=1&collection.configName=cdcr-target&name=cdcr-target&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2 and sendToOCPQueue=true
[junit4] 2> 30693 INFO (OverseerThreadFactory-65-thread-1-processing-n:127.0.0.1:58785_solr) [n:127.0.0.1:58785_solr ] o.a.s.c.CreateCollectionCmd Create collection cdcr-target
[junit4] 2> 30819 INFO (OverseerStateUpdate-98577368730763267-127.0.0.1:58785_solr-n_0000000000) [n:127.0.0.1:58785_solr ] o.a.s.c.o.SliceMutator createReplica() {
[junit4] 2> "operation":"ADDREPLICA",
[junit4] 2> "collection":"cdcr-target",
[junit4] 2> "shard":"shard1",
[junit4] 2> "core":"cdcr-target_shard1_replica_n1",
[junit4] 2> "state":"down",
[junit4] 2> "base_url":"http://127.0.0.1:58785/solr",
[junit4] 2> "type":"NRT"}
[junit4] 2> 31025 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr ] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node1&collection.configName=cdcr-target&newCollection=true&name=cdcr-target_shard1_replica_n1&action=CREATE&numShards=1&collection=cdcr-target&shard=shard1&wt=javabin&version=2&replicaType=NRT
[junit4] 2> 31026 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
[junit4] 2> 31146 INFO (zkCallback-32-thread-1-processing-n:127.0.0.1:58785_solr) [n:127.0.0.1:58785_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 31147 INFO (zkCallback-32-thread-2-processing-n:127.0.0.1:58785_solr) [n:127.0.0.1:58785_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 31172 INFO (OverseerCollectionConfigSetProcessor-98577368916819971-127.0.0.1:58814_solr-n_0000000000) [n:127.0.0.1:58814_solr ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist. Requestor may have disconnected from ZooKeeper
[junit4] 2> 32103 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.0.0
[junit4] 2> 32129 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.s.ManagedIndexSchemaFactory The schema is configured as managed, but managed schema resource managed-schema not found - loading non-managed schema schema.xml instead
[junit4] 2> 32144 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.s.IndexSchema [cdcr-target_shard1_replica_n1] Schema name=minimal
[junit4] 2> 32157 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
[junit4] 2> 32160 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.s.ManagedIndexSchema Created and persisted managed schema znode at /configs/cdcr-target/managed-schema
[junit4] 2> 32174 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.s.ManagedIndexSchemaFactory After upgrading to managed schema in ZooKeeper, renamed the non-managed schema /configs/cdcr-target/schema.xml to /configs/cdcr-target/schema.xml.bak
[junit4] 2> 32175 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'cdcr-target_shard1_replica_n1' using configuration from collection cdcr-target, trusted=true
[junit4] 2> 32177 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_58785.solr.core.cdcr-target.shard1.replica_n1' (registry 'solr.core.cdcr-target.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@18ab29c
[junit4] 2> 32183 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.SolrCore solr.RecoveryStrategy.Builder
[junit4] 2> 32183 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.SolrCore [[cdcr-target_shard1_replica_n1] ] Opening new SolrCore at [C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrBootstrapTest_243EB6F346B65BA1-001\cdcr-target-001\node1\cdcr-target_shard1_replica_n1], dataDir=[C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrBootstrapTest_243EB6F346B65BA1-001\cdcr-target-001\node1\.\cdcr-target_shard1_replica_n1\data\]
[junit4] 2> 32970 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.CdcrUpdateLog
[junit4] 2> 32972 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir= defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
[junit4] 2> 32983 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
[junit4] 2> 32983 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
[junit4] 2> 32995 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@bfc5b[cdcr-target_shard1_replica_n1] main]
[junit4] 2> 32997 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/cdcr-target
[junit4] 2> 32998 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/cdcr-target
[junit4] 2> 32998 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Creating ZooKeeper watch for the managed schema at /configs/cdcr-target/managed-schema
[junit4] 2> 33000 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.s.ZkIndexSchemaReader Current schema version 0 is already the latest
[junit4] 2> 33001 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000
[junit4] 2> 33007 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.CdcrBufferStateManager Created znode /collections/cdcr-target/cdcr/state/buffer
[junit4] 2> 33010 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.CdcrProcessStateManager Created znode /collections/cdcr-target/cdcr/state/process
[junit4] 2> 33013 INFO (searcherExecutor-82-thread-1-processing-n:127.0.0.1:58785_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-target_shard1_replica_n1] Registered new searcher Searcher@bfc5b[cdcr-target_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
[junit4] 2> 33013 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1577237910527148032
[junit4] 2> 33027 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
[junit4] 2> 33027 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
[junit4] 2> 33028 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.SyncStrategy Sync replicas to http://127.0.0.1:58785/solr/cdcr-target_shard1_replica_n1/
[junit4] 2> 33028 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.SyncStrategy Sync Success - now sync replicas to me
[junit4] 2> 33028 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.SyncStrategy http://127.0.0.1:58785/solr/cdcr-target_shard1_replica_n1/ has no replicas
[junit4] 2> 33028 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
[junit4] 2> 33038 INFO (zkCallback-32-thread-2-processing-n:127.0.0.1:58785_solr) [n:127.0.0.1:58785_solr ] o.a.s.h.CdcrLeaderStateManager Received new leader state @ cdcr-target:shard1
[junit4] 2> 33045 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.ShardLeaderElectionContext I am the new leader: http://127.0.0.1:58785/solr/cdcr-target_shard1_replica_n1/ shard1
[junit4] 2> 33157 INFO (zkCallback-32-thread-2-processing-n:127.0.0.1:58785_solr) [n:127.0.0.1:58785_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 33157 INFO (zkCallback-32-thread-1-processing-n:127.0.0.1:58785_solr) [n:127.0.0.1:58785_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 33203 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.ZkController I am the leader, no recovery necessary
[junit4] 2> 33206 INFO (qtp13866140-123) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node1&collection.configName=cdcr-target&newCollection=true&name=cdcr-target_shard1_replica_n1&action=CREATE&numShards=1&collection=cdcr-target&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2182
[junit4] 2> 33213 INFO (qtp13866140-120) [n:127.0.0.1:58785_solr ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
[junit4] 2> 33313 INFO (zkCallback-32-thread-2-processing-n:127.0.0.1:58785_solr) [n:127.0.0.1:58785_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 33313 INFO (zkCallback-32-thread-1-processing-n:127.0.0.1:58785_solr) [n:127.0.0.1:58785_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 34214 INFO (qtp13866140-120) [n:127.0.0.1:58785_solr ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={replicationFactor=1&collection.configName=cdcr-target&name=cdcr-target&nrtReplicas=1&action=CREATE&numShards=1&wt=javabin&version=2} status=0 QTime=3538
[junit4] 2> 34223 INFO (zkCallback-32-thread-1-processing-n:127.0.0.1:58785_solr) [n:127.0.0.1:58785_solr ] o.a.s.h.CdcrProcessStateManager The CDCR process state has changed: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/cdcr/state/process @ cdcr-target:shard1
[junit4] 2> 34223 INFO (qtp13866140-120) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&_stateVer_=cdcr-target:4&action=start&wt=javabin&version=2} status=0 QTime=5
[junit4] 2> 34223 INFO (zkCallback-32-thread-1-processing-n:127.0.0.1:58785_solr) [n:127.0.0.1:58785_solr ] o.a.s.h.CdcrProcessStateManager Received new CDCR process state from watcher: STARTED @ cdcr-target:shard1
[junit4] 2> 34235 INFO (qtp3048805-168) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
[junit4] 2> 34238 INFO (qtp3048805-168) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:58781/solr ready
[junit4] 2> 34251 INFO (qtp13866140-126) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Found maxVersionFromRecent 0 maxVersionFromIndex 0
[junit4] 2> 34251 INFO (qtp13866140-126) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/cdcr params={action=SHARDCHECKPOINT&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 34252 INFO (qtp13866140-124) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/cdcr params={_stateVer_=cdcr-target:4&action=COLLECTIONCHECKPOINT&wt=javabin&version=2} status=0 QTime=10
[junit4] 2> 34253 INFO (qtp3048805-168) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Create new update log reader for target cdcr-target with checkpoint -1 @ cdcr-source:shard1
[junit4] 2> 34253 INFO (qtp3048805-168) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-target, shard: shard1
[junit4] 2> 34256 INFO (qtp3048805-168) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Submitting bootstrap task to executor
[junit4] 2> 34257 INFO (zkCallback-44-thread-1-processing-n:127.0.0.1:58814_solr) [n:127.0.0.1:58814_solr ] o.a.s.h.CdcrProcessStateManager The CDCR process state has changed: WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-source/cdcr/state/process @ cdcr-source:shard1
[junit4] 2> 34257 INFO (qtp3048805-168) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&_stateVer_=cdcr-source:4&action=start&wt=javabin&version=2} status=0 QTime=31
[junit4] 2> 34257 INFO (zkCallback-44-thread-1-processing-n:127.0.0.1:58814_solr) [n:127.0.0.1:58814_solr ] o.a.s.h.CdcrProcessStateManager Received new CDCR process state from watcher: STARTED @ cdcr-source:shard1
[junit4] 2> 34259 INFO (cdcr-bootstrap-status-56-thread-1-processing-n:127.0.0.1:58814_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-target shard: shard1 leader: http://127.0.0.1:58785/solr/cdcr-target_shard1_replica_n1/
[junit4] 2> 34261 INFO (qtp3048805-167) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&_stateVer_=cdcr-source:4&action=queues&wt=javabin&version=2} status=0 QTime=3
[junit4] 1> Cdcr queue response: {responseHeader={status=0,QTime=3},queues={127.0.0.1:58781/solr={cdcr-target={queueSize=1010,lastTimestamp=}}},tlogTotalSize=47659,tlogTotalCount=10,updateLogSynchronizer=stopped}
[junit4] 2> 34263 INFO (qtp13866140-127) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&masterUrl=http://127.0.0.1:58814/solr/cdcr-source_shard1_replica_n1/&action=BOOTSTRAP&wt=javabin&version=2} status=0 QTime=1
[junit4] 2> 34263 INFO (qtp13866140-120) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577237911837868032,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 34264 INFO (qtp13866140-120) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 No uncommitted changes. Skipping IW.commit.
[junit4] 2> 34266 INFO (qtp13866140-120) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 34266 INFO (qtp13866140-120) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/update params={_stateVer_=cdcr-target:4&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} status=0 QTime=3
[junit4] 2> 34270 INFO (qtp13866140-125) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/select params={q=*:*&_stateVer_=cdcr-target:4&wt=javabin&version=2} hits=0 status=0 QTime=2
[junit4] 2> 34273 INFO (qtp13866140-126) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.S.Request [cdcr-target_shard1_replica_n1] webapp=/solr path=/cdcr params={qt=/cdcr&action=BOOTSTRAP_STATUS&wt=javabin&version=2} status=0 QTime=0
[junit4] 2> 34273 INFO (cdcr-bootstrap-status-56-thread-1-processing-n:127.0.0.1:58814_solr x:cdcr-source_shard1_replica_n1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.CdcrReplicatorManager CDCR bootstrap running for 1 seconds, sleeping for 2000 ms
[junit4] 2> 34274 INFO (recoveryExecutor-30-thread-1-processing-n:127.0.0.1:58785_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.u.UpdateLog Starting to buffer updates. FSUpdateLog{state=ACTIVE, tlog=null}
[junit4] 2> 34278 INFO (qtp3048805-166) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1577237911853596672,optimize=false,openSearcher=false,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
[junit4] 2> 34278 INFO (qtp3048805-166) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@1779ed5 commitCommandVersion:1577237911853596672
[junit4] 2> 34296 INFO (qtp3048805-166) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@187108e[cdcr-source_shard1_replica_n1] realtime]
[junit4] 2> 34296 INFO (qtp3048805-166) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.u.DirectUpdateHandler2 end_commit_flush
[junit4] 2> 34297 INFO (qtp3048805-166) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/update params={waitSearcher=true&openSearcher=false&commit=true&softCommit=false&commit_end_point=true&wt=javabin&version=2} status=0 QTime=19
[junit4] 2> 34311 INFO (qtp3048805-164) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
[junit4] 2> 34311 INFO (recoveryExecutor-30-thread-1-processing-n:127.0.0.1:58785_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.IndexFetcher Master's generation: 12
[junit4] 2> 34312 INFO (recoveryExecutor-30-thread-1-processing-n:127.0.0.1:58785_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.IndexFetcher Master's version: 1504171287397
[junit4] 2> 34312 INFO (recoveryExecutor-30-thread-1-processing-n:127.0.0.1:58785_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.IndexFetcher Slave's generation: 1
[junit4] 2> 34312 INFO (recoveryExecutor-30-thread-1-processing-n:127.0.0.1:58785_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.IndexFetcher Slave's version: 0
[junit4] 2> 34312 INFO (recoveryExecutor-30-thread-1-processing-n:127.0.0.1:58785_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.IndexFetcher Starting replication process
[junit4] 2> 34323 INFO (qtp3048805-163) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.h.ReplicationHandler Adding tlog files to list: [{size=4657, name=tlog.0000000000000000000.1577237907069992960}, {size=4778, name=tlog.0000000000000000001.1577237907171704832}, {size=4778, name=tlog.0000000000000000002.1577237907252445184}, {size=4778, name=tlog.0000000000000000003.1577237907314311168}, {size=4778, name=tlog.0000000000000000004.1577237907386662912}, {size=4778, name=tlog.0000000000000000005.1577237907512492032}, {size=4778, name=tlog.0000000000000000006.1577237907591135232}, {size=4778, name=tlog.0000000000000000007.1577237907666632704}, {size=4778, name=tlog.0000000000000000008.1577237907755761664}, {size=4778, name=tlog.0000000000000000009.1577237907854327808}]
[junit4] 2> 34323 INFO (qtp3048805-163) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&wt=javabin&version=2&command=filelist} status=0 QTime=10
[junit4] 2> 34324 INFO (recoveryExecutor-30-thread-1-processing-n:127.0.0.1:58785_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.IndexFetcher Number of files in latest index in master: 13
[junit4] 2> 34324 INFO (recoveryExecutor-30-thread-1-processing-n:127.0.0.1:58785_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.IndexFetcher Number of tlog files in master: 10
[junit4] 2> 34328 INFO (recoveryExecutor-30-thread-1-processing-n:127.0.0.1:58785_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
[junit4] 2> 34328 INFO (recoveryExecutor-30-thread-1-processing-n:127.0.0.1:58785_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MockDirectoryWrapper(RAMDirectory@44ccf7 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@11b8804)
[junit4] 2> 34349 INFO (qtp3048805-168) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_Lucene50_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=7
[junit4] 2> 34357 INFO (qtp3048805-167) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 34361 INFO (qtp3048805-161) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_Lucene50_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 34363 INFO (qtp3048805-166) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_Lucene50_0.tim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 34370 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 34372 INFO (qtp3048805-164) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 34378 INFO (qtp3048805-163) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a_Lucene50_0.tip&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 34380 INFO (qtp3048805-168) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.nvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 34385 INFO (qtp3048805-167) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.fnm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 34387 INFO (qtp3048805-167) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.dii&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 34391 INFO (qtp3048805-166) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.dim&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 34393 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=_a.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 34397 INFO (qtp3048805-164) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&file=segments_c&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 34398 INFO (recoveryExecutor-30-thread-1-processing-n:127.0.0.1:58785_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.IndexFetcher Starting download of tlog files from master: [{size=4657, name=tlog.0000000000000000000.1577237907069992960}, {size=4778, name=tlog.0000000000000000001.1577237907171704832}, {size=4778, name=tlog.0000000000000000002.1577237907252445184}, {size=4778, name=tlog.0000000000000000003.1577237907314311168}, {size=4778, name=tlog.0000000000000000004.1577237907386662912}, {size=4778, name=tlog.0000000000000000005.1577237907512492032}, {size=4778, name=tlog.0000000000000000006.1577237907591135232}, {size=4778, name=tlog.0000000000000000007.1577237907666632704}, {size=4778, name=tlog.0000000000000000008.1577237907755761664}, {size=4778, name=tlog.0000000000000000009.1577237907854327808}]
[junit4] 2> 34428 INFO (qtp3048805-163) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000000.1577237907069992960&wt=filestream&command=filecontent} status=0 QTime=20
[junit4] 2> 34432 INFO (qtp3048805-168) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000001.1577237907171704832&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 34438 INFO (qtp3048805-161) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000002.1577237907252445184&wt=filestream&command=filecontent} status=0 QTime=1
[junit4] 2> 34441 INFO (qtp3048805-167) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000003.1577237907314311168&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 34445 INFO (qtp3048805-166) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000004.1577237907386662912&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 34450 INFO (qtp3048805-165) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000005.1577237907512492032&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 34453 INFO (qtp3048805-164) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000006.1577237907591135232&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 34456 INFO (qtp3048805-163) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000007.1577237907666632704&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 34461 INFO (qtp3048805-168) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000008.1577237907755761664&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 34467 INFO (qtp3048805-161) [n:127.0.0.1:58814_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica_n1] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&checksum=true&tlogFile=tlog.0000000000000000009.1577237907854327808&wt=filestream&command=filecontent} status=0 QTime=0
[junit4] 2> 34468 INFO (recoveryExecutor-30-thread-1-processing-n:127.0.0.1:58785_solr x:cdcr-target_shard1_replica_n1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:58785_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.IndexFetcher Total time taken for download (fullCopy=false,bytesDownloaded=104436) : 0 secs (null bytes/sec) to MockDirectoryWrapper(RAMDire
[...truncated too long message...]
f16b{/solr,null,UNAVAILABLE}
[junit4] 2> 184562 ERROR (TEST-CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster-seed#[243EB6F346B65BA1]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 184563 INFO (TEST-CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster-seed#[243EB6F346B65BA1]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:59101 59101
[junit4] 2> 184564 INFO (Thread-79) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:59101 59101
[junit4] 2> 184565 WARN (Thread-79) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/aliases.json
[junit4] 2> 2 /solr/clusterprops.json
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/collections/cdcr-source/state.json
[junit4] 2> 2 /solr/clusterstate.json
[junit4] 2> 2 /solr/collections/cdcr-source/cdcr/state/process
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 2 /solr/live_nodes
[junit4] 2> 2 /solr/collections
[junit4] 2>
[junit4] 2> 184566 INFO (jetty-closer-102-thread-1) [ ] o.e.j.s.AbstractConnector Stopped ServerConnector@1dc248{HTTP/1.1,[http/1.1]}{127.0.0.1:0}
[junit4] 2> 184568 INFO (jetty-closer-102-thread-1) [ ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=16166634
[junit4] 2> 184568 INFO (jetty-closer-102-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
[junit4] 2> 184568 INFO (jetty-closer-102-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@9b035d: rootName = solr_59081, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@d87a3c
[junit4] 2> 184575 INFO (jetty-closer-102-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
[junit4] 2> 184576 INFO (jetty-closer-102-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5c8d5b: rootName = solr_59081, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@13e2510
[junit4] 2> 184583 INFO (jetty-closer-102-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
[junit4] 2> 184583 INFO (jetty-closer-102-thread-1) [ ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@19cf535: rootName = solr_59081, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@13b7476
[junit4] 2> 184585 INFO (jetty-closer-102-thread-1) [ ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
[junit4] 2> 184588 INFO (coreCloseExecutor-216-thread-1) [n:127.0.0.1:59081_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.c.SolrCore [cdcr-target_shard1_replica_n1] CLOSING SolrCore org.apache.solr.core.SolrCore@148eef
[junit4] 2> 184588 INFO (coreCloseExecutor-216-thread-1) [n:127.0.0.1:59081_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.cdcr-target.shard1.replica_n1, tag=1347311
[junit4] 2> 184588 INFO (coreCloseExecutor-216-thread-1) [n:127.0.0.1:59081_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5c8f14: rootName = solr_59081, domain = solr.core.cdcr-target.shard1.replica_n1, service url = null, agent id = null] for registry solr.core.cdcr-target.shard1.replica_n1 / com.codahale.metrics.MetricRegistry@1e3c798
[junit4] 2> 184590 INFO (zkCallback-108-thread-3-processing-n:127.0.0.1:59081_solr) [n:127.0.0.1:59081_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 184591 INFO (zkCallback-108-thread-4-processing-n:127.0.0.1:59081_solr) [n:127.0.0.1:59081_solr ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/cdcr-target/state.json] for collection [cdcr-target] has occurred - updating... (live nodes size: [1])
[junit4] 2> 184604 INFO (coreCloseExecutor-216-thread-1) [n:127.0.0.1:59081_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.collection.cdcr-target.shard1.leader, tag=1347311
[junit4] 2> 184605 INFO (coreCloseExecutor-216-thread-1) [n:127.0.0.1:59081_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica_n1] o.a.s.h.CdcrRequestHandler Solr core is being closed - shutting down CDCR handler @ cdcr-target:shard1
[junit4] 2> 184608 INFO (jetty-closer-102-thread-1) [ ] o.a.s.c.Overseer Overseer (id=98577370827325443-127.0.0.1:59081_solr-n_0000000000) closing
[junit4] 2> 184608 INFO (OverseerStateUpdate-98577370827325443-127.0.0.1:59081_solr-n_0000000000) [n:127.0.0.1:59081_solr ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:59081_solr
[junit4] 2> 184611 WARN (zkCallback-108-thread-4-processing-n:127.0.0.1:59081_solr) [n:127.0.0.1:59081_solr ] o.a.s.c.c.ZkStateReader ZooKeeper watch triggered, but Solr cannot talk to ZK: [KeeperErrorCode = Session expired for /live_nodes]
[junit4] 2> 184612 INFO (jetty-closer-102-thread-1) [ ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@47956a{/solr,null,UNAVAILABLE}
[junit4] 2> 184613 ERROR (TEST-CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster-seed#[243EB6F346B65BA1]) [ ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
[junit4] 2> 184613 INFO (TEST-CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster-seed#[243EB6F346B65BA1]) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:59077 59077
[junit4] 2> 184615 INFO (Thread-69) [ ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:59077 59077
[junit4] 2> 184616 WARN (Thread-69) [ ] o.a.s.c.ZkTestServer Watch limit violations:
[junit4] 2> Maximum concurrent create/delete watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/aliases.json
[junit4] 2> 3 /solr/clusterprops.json
[junit4] 2> 2 /solr/configs/cdcr-target/managed-schema
[junit4] 2> 2 /solr/collections/cdcr-target/leaders/shard1/leader
[junit4] 2>
[junit4] 2> Maximum concurrent data watches above limit:
[junit4] 2>
[junit4] 2> 6 /solr/collections/cdcr-target/state.json
[junit4] 2> 3 /solr/collections/cdcr-target/cdcr/state/process
[junit4] 2> 3 /solr/clusterstate.json
[junit4] 2> 2 /solr/collections/cdcr-target/cdcr/state/buffer
[junit4] 2>
[junit4] 2> Maximum concurrent children watches above limit:
[junit4] 2>
[junit4] 2> 3 /solr/live_nodes
[junit4] 2> 3 /solr/collections
[junit4] 2>
[junit4] 2> 184616 INFO (TEST-CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster-seed#[243EB6F346B65BA1]) [ ] o.a.s.SolrTestCaseJ4 ###Ending testBootstrapWithContinousIndexingOnSourceCluster
[junit4] 2> NOTE: reproduce with: ant test -Dtestcase=CdcrBootstrapTest -Dtests.method=testBootstrapWithContinousIndexingOnSourceCluster -Dtests.seed=243EB6F346B65BA1 -Dtests.slow=true -Dtests.locale=en-SG -Dtests.timezone=Asia/Seoul -Dtests.asserts=true -Dtests.file.encoding=Cp1252
[junit4] FAILURE 130s J0 | CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster <<<
[junit4] > Throwable #1: java.lang.AssertionError: Document mismatch on target after sync expected:<2000> but was:<1100>
[junit4] > at __randomizedtesting.SeedInfo.seed([243EB6F346B65BA1:F07BFDAAA1E0E85A]:0)
[junit4] > at org.apache.solr.cloud.CdcrBootstrapTest.testBootstrapWithContinousIndexingOnSourceCluster(CdcrBootstrapTest.java:309)
[junit4] > at java.lang.Thread.run(Thread.java:748)
[junit4] 2> NOTE: leaving temporary files on disk at: C:\Users\jenkins\workspace\Lucene-Solr-7.0-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.CdcrBootstrapTest_243EB6F346B65BA1-001
[junit4] 2> Aug 31, 2017 9:23:57 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
[junit4] 2> WARNING: Will linger awaiting termination of 5 leaked thread(s).
[junit4] 2> NOTE: test params are: codec=DummyCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=DUMMY, chunkSize=17107, maxDocsPerChunk=672, blockSize=5), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=DUMMY, chunkSize=17107, blockSize=5)), sim=RandomSimilarity(queryNorm=true): {}, locale=en-SG, timezone=Asia/Seoul
[junit4] 2> NOTE: Windows 10 10.0 x86/Oracle Corporation 1.8.0_144 (32-bit)/cpus=3,threads=1,free=25722752,total=68538368
[junit4] 2> NOTE: All tests run in this JVM: [BadCopyFieldTest, ZkControllerTest, TestNRTOpen, BigEndianAscendingWordSerializerTest, TestSchemalessBufferedUpdates, SolrMetricManagerTest, CdcrBootstrapTest]
[junit4] Completed [19/729 (1!)] on J0 in 162.55s, 3 tests, 1 failure <<< FAILURES!
[...truncated 41775 lines...]