You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2018/10/16 01:50:44 UTC

[JENKINS] Lucene-Solr-7.x-MacOSX (64bit/jdk-9) - Build # 886 - Still Unstable!

Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-MacOSX/886/
Java: 64bit/jdk-9 -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC

11 tests failed.
FAILED:  org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure

Error Message:
Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$117/783933817@401ee6d2 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@7a2fb087[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 9]

Stack Trace:
java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$117/783933817@401ee6d2 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@7a2fb087[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 9]
	at __randomizedtesting.SeedInfo.seed([D27969BD6D3111F3:5A4FCBEEB59EF9E1]:0)
	at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2104)
	at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:848)
	at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1397)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:194)
	at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140)
	at org.apache.solr.common.cloud.TestCollectionStateWatchers.waitInBackground(TestCollectionStateWatchers.java:74)
	at org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure(TestCollectionStateWatchers.java:241)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.common.cloud.TestCollectionStateWatchers.testStateWatcherChecksCurrentStateOnRegister

Error Message:
Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$117/783933817@46f2e83c rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@7a2fb087[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 9]

Stack Trace:
java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$117/783933817@46f2e83c rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@7a2fb087[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 9]
	at __randomizedtesting.SeedInfo.seed([D27969BD6D3111F3:BC7D5683043F50B1]:0)
	at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2104)
	at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:848)
	at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1397)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:194)
	at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140)
	at org.apache.solr.common.cloud.TestCollectionStateWatchers.waitFor(TestCollectionStateWatchers.java:86)
	at org.apache.solr.common.cloud.TestCollectionStateWatchers.testStateWatcherChecksCurrentStateOnRegister(TestCollectionStateWatchers.java:173)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.common.cloud.TestCollectionStateWatchers.testPredicateFailureTimesOut

Error Message:
Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$117/783933817@6162791e rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@7a2fb087[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 9]

Stack Trace:
java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$117/783933817@6162791e rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@7a2fb087[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 9]
	at __randomizedtesting.SeedInfo.seed([D27969BD6D3111F3:2F400F813429C489]:0)
	at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2104)
	at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:848)
	at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1397)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:194)
	at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140)
	at org.apache.solr.common.cloud.TestCollectionStateWatchers.waitFor(TestCollectionStateWatchers.java:86)
	at org.apache.solr.common.cloud.TestCollectionStateWatchers.testPredicateFailureTimesOut(TestCollectionStateWatchers.java:220)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.common.cloud.TestCollectionStateWatchers.testWatcherIsRemovedAfterTimeout

Error Message:
Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$117/783933817@2f694330 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@7a2fb087[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 9]

Stack Trace:
java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$117/783933817@2f694330 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@7a2fb087[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 9]
	at __randomizedtesting.SeedInfo.seed([D27969BD6D3111F3:F3F34C8DB3C138C9]:0)
	at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2104)
	at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:848)
	at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1397)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:194)
	at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140)
	at org.apache.solr.common.cloud.TestCollectionStateWatchers.waitFor(TestCollectionStateWatchers.java:86)
	at org.apache.solr.common.cloud.TestCollectionStateWatchers.testWatcherIsRemovedAfterTimeout(TestCollectionStateWatchers.java:270)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.common.cloud.TestCollectionStateWatchers.testDeletionsTriggerWatches

Error Message:
Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$117/783933817@6f532eae rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@7a2fb087[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 9]

Stack Trace:
java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$117/783933817@6f532eae rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@7a2fb087[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 9]
	at __randomizedtesting.SeedInfo.seed([D27969BD6D3111F3:70B8A132EF52C79E]:0)
	at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2104)
	at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:848)
	at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1397)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:194)
	at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140)
	at org.apache.solr.common.cloud.TestCollectionStateWatchers.waitInBackground(TestCollectionStateWatchers.java:74)
	at org.apache.solr.common.cloud.TestCollectionStateWatchers.testDeletionsTriggerWatches(TestCollectionStateWatchers.java:280)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure

Error Message:
Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$117/1259779393@79da60d1 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@25508574[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8]

Stack Trace:
java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$117/1259779393@79da60d1 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@25508574[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8]
	at __randomizedtesting.SeedInfo.seed([D27969BD6D3111F3:5A4FCBEEB59EF9E1]:0)
	at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2104)
	at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:848)
	at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1397)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:194)
	at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140)
	at org.apache.solr.common.cloud.TestCollectionStateWatchers.waitInBackground(TestCollectionStateWatchers.java:74)
	at org.apache.solr.common.cloud.TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure(TestCollectionStateWatchers.java:241)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.common.cloud.TestCollectionStateWatchers.testStateWatcherChecksCurrentStateOnRegister

Error Message:
Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$117/1259779393@68ab3ea1 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@25508574[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8]

Stack Trace:
java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$117/1259779393@68ab3ea1 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@25508574[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8]
	at __randomizedtesting.SeedInfo.seed([D27969BD6D3111F3:BC7D5683043F50B1]:0)
	at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2104)
	at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:848)
	at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1397)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:194)
	at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140)
	at org.apache.solr.common.cloud.TestCollectionStateWatchers.waitFor(TestCollectionStateWatchers.java:86)
	at org.apache.solr.common.cloud.TestCollectionStateWatchers.testStateWatcherChecksCurrentStateOnRegister(TestCollectionStateWatchers.java:173)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.common.cloud.TestCollectionStateWatchers.testPredicateFailureTimesOut

Error Message:
Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$117/1259779393@55d25637 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@25508574[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8]

Stack Trace:
java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$117/1259779393@55d25637 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@25508574[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8]
	at __randomizedtesting.SeedInfo.seed([D27969BD6D3111F3:2F400F813429C489]:0)
	at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2104)
	at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:848)
	at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1397)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:194)
	at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140)
	at org.apache.solr.common.cloud.TestCollectionStateWatchers.waitFor(TestCollectionStateWatchers.java:86)
	at org.apache.solr.common.cloud.TestCollectionStateWatchers.testPredicateFailureTimesOut(TestCollectionStateWatchers.java:220)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.common.cloud.TestCollectionStateWatchers.testWatcherIsRemovedAfterTimeout

Error Message:
Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$117/1259779393@2ca2cb16 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@25508574[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8]

Stack Trace:
java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$117/1259779393@2ca2cb16 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@25508574[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8]
	at __randomizedtesting.SeedInfo.seed([D27969BD6D3111F3:F3F34C8DB3C138C9]:0)
	at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2104)
	at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:848)
	at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1397)
	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:194)
	at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140)
	at org.apache.solr.common.cloud.TestCollectionStateWatchers.waitFor(TestCollectionStateWatchers.java:86)
	at org.apache.solr.common.cloud.TestCollectionStateWatchers.testWatcherIsRemovedAfterTimeout(TestCollectionStateWatchers.java:270)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.common.cloud.TestCollectionStateWatchers.testSimpleCollectionWatch

Error Message:
CollectionStateWatcher was never notified of cluster change

Stack Trace:
java.lang.AssertionError: CollectionStateWatcher was never notified of cluster change
	at __randomizedtesting.SeedInfo.seed([D27969BD6D3111F3:8F42A6CD2A3C8ECD]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.common.cloud.TestCollectionStateWatchers.testSimpleCollectionWatch(TestCollectionStateWatchers.java:140)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.common.cloud.TestCollectionStateWatchers.testSimpleCollectionWatch

Error Message:
CollectionStateWatcher was never notified of cluster change

Stack Trace:
java.lang.AssertionError: CollectionStateWatcher was never notified of cluster change
	at __randomizedtesting.SeedInfo.seed([D27969BD6D3111F3:8F42A6CD2A3C8ECD]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.common.cloud.TestCollectionStateWatchers.testSimpleCollectionWatch(TestCollectionStateWatchers.java:140)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 1917 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/core/test/temp/junit4-J0-20181015_235840_0633809899301358906186.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 5 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/core/test/temp/junit4-J1-20181015_235840_0635046541906299550718.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 293 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/test-framework/test/temp/junit4-J1-20181016_000513_6629867981128253057928.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/test-framework/test/temp/junit4-J0-20181016_000513_66213154774073454377557.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 1080 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/analysis/common/test/temp/junit4-J0-20181016_000616_21116411642649559128623.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/analysis/common/test/temp/junit4-J1-20181016_000616_2113188485839255655113.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 256 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/analysis/icu/test/temp/junit4-J0-20181016_000912_11412742246030143446930.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/analysis/icu/test/temp/junit4-J1-20181016_000912_12314714805656782867391.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 253 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/analysis/kuromoji/test/temp/junit4-J1-20181016_000930_1118487734867718101745.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/analysis/kuromoji/test/temp/junit4-J0-20181016_000930_11170874130123385038.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 162 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/analysis/morfologik/test/temp/junit4-J1-20181016_001011_00218066792449748040391.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/analysis/morfologik/test/temp/junit4-J0-20181016_001011_0027727789818849731568.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 207 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/analysis/nori/test/temp/junit4-J0-20181016_001018_15311128427602821992686.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/analysis/nori/test/temp/junit4-J1-20181016_001018_1539601249218633259254.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 171 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/analysis/opennlp/test/temp/junit4-J0-20181016_001034_11116960923571806766569.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/analysis/opennlp/test/temp/junit4-J1-20181016_001034_11612675217741576411876.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 176 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/analysis/phonetic/test/temp/junit4-J1-20181016_001040_57417116979852973229258.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/analysis/phonetic/test/temp/junit4-J0-20181016_001040_5744506929582850957704.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 159 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/analysis/smartcn/test/temp/junit4-J0-20181016_001054_6175672492826110795953.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/analysis/smartcn/test/temp/junit4-J1-20181016_001054_6175505078946803079181.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 167 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/analysis/stempel/test/temp/junit4-J0-20181016_001106_8396881156500370386681.syserr
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/analysis/stempel/test/temp/junit4-J1-20181016_001106_8396053861423408974536.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 225 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/backward-codecs/test/temp/junit4-J1-20181016_001114_85313585666139005111337.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 9 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/backward-codecs/test/temp/junit4-J0-20181016_001114_85317415304056075224235.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 1423 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/benchmark/test/temp/junit4-J0-20181016_001340_25512326725064172285370.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/benchmark/test/temp/junit4-J1-20181016_001340_2568891000172184290317.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 251 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/classification/test/temp/junit4-J0-20181016_001355_1918688616629707356125.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/classification/test/temp/junit4-J1-20181016_001355_1915698674758062184117.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 267 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/codecs/test/temp/junit4-J0-20181016_001419_65917462776060162265888.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/codecs/test/temp/junit4-J1-20181016_001419_65917082446994959680673.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 234 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/demo/test/temp/junit4-J0-20181016_001644_3758725765479218199148.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/demo/test/temp/junit4-J1-20181016_001644_3754641076508380457449.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 173 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/expressions/test/temp/junit4-J0-20181016_001651_6201326962778560862145.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/expressions/test/temp/junit4-J1-20181016_001651_6275778528844304647187.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 234 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/facet/test/temp/junit4-J1-20181016_001702_2106654746558412343902.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/facet/test/temp/junit4-J0-20181016_001702_2094650132190020257921.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 182 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/grouping/test/temp/junit4-J0-20181016_001753_8566535924958327849074.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/grouping/test/temp/junit4-J1-20181016_001753_85612453204978769052336.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 251 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/highlighter/test/temp/junit4-J1-20181016_001811_02718177303848270532568.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/highlighter/test/temp/junit4-J0-20181016_001811_0132446473939929674020.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 166 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/join/test/temp/junit4-J0-20181016_001846_96516766413964525195227.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/join/test/temp/junit4-J1-20181016_001846_97117128368315152495073.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 155 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/memory/test/temp/junit4-J0-20181016_001904_9326207846876666716816.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/memory/test/temp/junit4-J1-20181016_001904_93314472402958357659142.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 194 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/misc/test/temp/junit4-J1-20181016_001914_55416584281933504360677.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/misc/test/temp/junit4-J0-20181016_001914_5544094814407653821980.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 338 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/queries/test/temp/junit4-J1-20181016_001941_14413891218532068486805.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/queries/test/temp/junit4-J0-20181016_001941_13911628890069220831492.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 231 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/queryparser/test/temp/junit4-J1-20181016_001956_7145404135675333268527.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/queryparser/test/temp/junit4-J0-20181016_001956_71414705258937888425242.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 196 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/replicator/test/temp/junit4-J1-20181016_002015_8508605503006934465463.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 23 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/replicator/test/temp/junit4-J0-20181016_002015_8508664063794881058595.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 213 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/sandbox/test/temp/junit4-J1-20181016_002035_2414807698893539750406.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/sandbox/test/temp/junit4-J0-20181016_002035_2387014464157489842424.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 299 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/spatial-extras/test/temp/junit4-J1-20181016_002144_99012878697222650616345.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 3 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/spatial-extras/test/temp/junit4-J0-20181016_002144_99013711726761877065707.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 151 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/spatial/test/temp/junit4-J0-20181016_002234_2671730247081709176499.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 177 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/spatial3d/test/temp/junit4-J0-20181016_002237_109723310751959940328.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 14 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/spatial3d/test/temp/junit4-J1-20181016_002237_1094558711260688740434.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 252 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/suggest/test/temp/junit4-J0-20181016_002300_7162528126441303373982.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 12 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/build/suggest/test/temp/junit4-J1-20181016_002300_71613864044807315656768.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 4762 lines...]
   [junit4] JVM J1: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/temp/junit4-J1-20181016_002456_7047968871565542048550.syserr
   [junit4] >>> JVM J1 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J1: EOF ----

[...truncated 4 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/temp/junit4-J0-20181016_002456_7036181599183472041505.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 718 lines...]
   [junit4] Suite: org.apache.solr.common.cloud.TestCollectionStateWatchers
   [junit4]   2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_D27969BD6D3111F3-001/init-core-data-001
   [junit4]   1> 01:33:43.503 [Thread-95] ERROR org.apache.zookeeper.server.ZooKeeperServer - ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   1> 01:33:43.636 [jetty-launcher-1029-thread-2] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 01:33:43.636 [jetty-launcher-1029-thread-4] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 01:33:43.638 [jetty-launcher-1029-thread-3] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 01:33:43.640 [jetty-launcher-1029-thread-1] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 01:33:48.642 [TEST-TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure-seed#[D27969BD6D3111F3]] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 01:33:50.351 [TEST-TestCollectionStateWatchers.testWaitForStateWatcherIsRetainedOnPredicateFailure-seed#[D27969BD6D3111F3]] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestCollectionStateWatchers -Dtests.method=testSimpleCollectionWatch -Dtests.seed=D27969BD6D3111F3 -Dtests.slow=true -Dtests.locale=ru-KG -Dtests.timezone=America/Indiana/Tell_City -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] FAILURE 33.2s J1 | TestCollectionStateWatchers.testSimpleCollectionWatch <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: CollectionStateWatcher was never notified of cluster change
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([D27969BD6D3111F3:8F42A6CD2A3C8ECD]:0)
   [junit4]    > 	at org.apache.solr.common.cloud.TestCollectionStateWatchers.testSimpleCollectionWatch(TestCollectionStateWatchers.java:140)
   [junit4]    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]    > 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]    > 	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
   [junit4]    > 	at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   1> 01:34:40.633 [TEST-TestCollectionStateWatchers.testWaitForStateChecksCurrentState-seed#[D27969BD6D3111F3]] ERROR org.apache.solr.util.StartupLoggingUtils - Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   1> 01:34:45.999 [SUITE-TestCollectionStateWatchers-seed#[D27969BD6D3111F3]-worker] ERROR org.apache.zookeeper.server.ZooKeeperServer - ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-solrj/test/J1/temp/solr.common.cloud.TestCollectionStateWatchers_D27969BD6D3111F3-001
   [junit4]   2> Oct 16, 2018 1:34:46 AM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=true): {}, locale=ru-KG, timezone=America/Indiana/Tell_City
   [j

[...truncated too long message...]

/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2104)
   [junit4]    > 	at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:848)
   [junit4]    > 	at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1397)
   [junit4]    > 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:194)
   [junit4]    > 	at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140)
   [junit4]    > 	at org.apache.solr.common.cloud.TestCollectionStateWatchers.waitFor(TestCollectionStateWatchers.java:86)
   [junit4]    > 	at org.apache.solr.common.cloud.TestCollectionStateWatchers.testStateWatcherChecksCurrentStateOnRegister(TestCollectionStateWatchers.java:173)
   [junit4]    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]    > 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]    > 	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
   [junit4]    > 	at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestCollectionStateWatchers -Dtests.method=testPredicateFailureTimesOut -Dtests.seed=D27969BD6D3111F3 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=ru-KG -Dtests.timezone=America/Indiana/Tell_City -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   1.07s J0 | TestCollectionStateWatchers.testPredicateFailureTimesOut <<<
   [junit4]    > Throwable #1: java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$117/1259779393@55d25637 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@25508574[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([D27969BD6D3111F3:2F400F813429C489]:0)
   [junit4]    > 	at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2104)
   [junit4]    > 	at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:848)
   [junit4]    > 	at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1397)
   [junit4]    > 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:194)
   [junit4]    > 	at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140)
   [junit4]    > 	at org.apache.solr.common.cloud.TestCollectionStateWatchers.waitFor(TestCollectionStateWatchers.java:86)
   [junit4]    > 	at org.apache.solr.common.cloud.TestCollectionStateWatchers.testPredicateFailureTimesOut(TestCollectionStateWatchers.java:220)
   [junit4]    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]    > 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]    > 	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
   [junit4]    > 	at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestCollectionStateWatchers -Dtests.method=testWatcherIsRemovedAfterTimeout -Dtests.seed=D27969BD6D3111F3 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=ru-KG -Dtests.timezone=America/Indiana/Tell_City -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4] ERROR   0.08s J0 | TestCollectionStateWatchers.testWatcherIsRemovedAfterTimeout <<<
   [junit4]    > Throwable #1: java.util.concurrent.RejectedExecutionException: Task org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$$Lambda$117/1259779393@2ca2cb16 rejected from org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor@25508574[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 8]
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([D27969BD6D3111F3:F3F34C8DB3C138C9]:0)
   [junit4]    > 	at java.base/java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2104)
   [junit4]    > 	at java.base/java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:848)
   [junit4]    > 	at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1397)
   [junit4]    > 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.execute(ExecutorUtil.java:194)
   [junit4]    > 	at java.base/java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:140)
   [junit4]    > 	at org.apache.solr.common.cloud.TestCollectionStateWatchers.waitFor(TestCollectionStateWatchers.java:86)
   [junit4]    > 	at org.apache.solr.common.cloud.TestCollectionStateWatchers.testWatcherIsRemovedAfterTimeout(TestCollectionStateWatchers.java:270)
   [junit4]    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]    > 	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]    > 	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]    > 	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
   [junit4]    > 	at java.base/java.lang.Thread.run(Thread.java:844)
   [junit4] IGNOR/A 0.05s J0 | TestCollectionStateWatchers.testDeletionsTriggerWatches
   [junit4]    > Assumption #1: Ignored, failures limit reached (5 >= 5).
   [junit4] IGNOR/A 0.05s J0 | TestCollectionStateWatchers.testCanWaitForNonexistantCollection
   [junit4]    > Assumption #1: Ignored, failures limit reached (5 >= 5).
   [junit4] IGNOR/A 0.06s J0 | TestCollectionStateWatchers.testWatchesWorkForStateFormat1
   [junit4]    > Assumption #1: Ignored, failures limit reached (5 >= 5).
   [junit4] IGNOR/A 0.05s J0 | TestCollectionStateWatchers.testSimpleCollectionWatch
   [junit4]    > Assumption #1: Ignored, failures limit reached (5 >= 5).
   [junit4] IGNOR/A 0.06s J0 | TestCollectionStateWatchers.testWaitForStateChecksCurrentState
   [junit4]    > Assumption #1: Ignored, failures limit reached (5 >= 5).
   [junit4]   1> 01:47:19.577 [SUITE-TestCollectionStateWatchers-seed#[D27969BD6D3111F3]-worker] ERROR org.apache.zookeeper.server.ZooKeeperServer - ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-solrj/test/J0/temp/solr.common.cloud.TestCollectionStateWatchers_D27969BD6D3111F3-002
   [junit4]   2> NOTE: test params are: codec=CheapBastard, sim=RandomSimilarity(queryNorm=true): {}, locale=ru-KG, timezone=America/Indiana/Tell_City
   [junit4]   2> NOTE: Mac OS X 10.11.6 x86_64/Oracle Corporation 9 (64-bit)/cpus=3,threads=1,free=68880608,total=119164928
   [junit4]   2> NOTE: All tests run in this JVM: [TestCollectionStateWatchers, TestCollectionStateWatchers]
   [junit4] Completed [5/5 (3!)] on J0 in 9.87s, 9 tests, 4 errors, 5 skipped <<< FAILURES!

[...truncated 1 lines...]
   [junit4] JVM J0: stderr was not empty, see: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-solrj/test/temp/junit4-J0-20181016_014600_2041148477225383293903.syserr
   [junit4] >>> JVM J0 emitted unexpected output (verbatim) ----
   [junit4] Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
   [junit4] <<< JVM J0: EOF ----

[...truncated 18 lines...]
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/common-build.xml:1567: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/common-build.xml:1092: There were test failures: 5 suites, 45 tests, 9 errors, 1 failure, 18 ignored (9 assumptions) [seed: D27969BD6D3111F3]

Total time: 1 minute 21 seconds

[repro] Setting last failure code to 256

[repro] Failures:
[repro]   3/5 failed: org.apache.solr.common.cloud.TestCollectionStateWatchers
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=653894fe2028107a090f44f9f46aa0bdf7d1424b, workspace=/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX
[WARNINGS] Computing warning deltas based on reference build #885
Recording test results
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2

[JENKINS] Lucene-Solr-7.x-MacOSX (64bit/jdk-9) - Build # 888 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-MacOSX/888/
Java: 64bit/jdk-9 -XX:-UseCompressedOops -XX:+UseSerialGC

5 tests failed.
FAILED:  org.apache.solr.cloud.TestTlogReplica.testRecovery

Error Message:
Can not find doc 7 in https://127.0.0.1:58238/solr

Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:58238/solr
	at __randomizedtesting.SeedInfo.seed([3B77B4AB27EAD1B:C24702E69F2E67BC]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
	at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestTlogReplica.testRecovery

Error Message:
Can not find doc 7 in https://127.0.0.1:58552/solr

Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:58552/solr
	at __randomizedtesting.SeedInfo.seed([3B77B4AB27EAD1B:C24702E69F2E67BC]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
	at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestTlogReplica.testRecovery

Error Message:
Can not find doc 7 in https://127.0.0.1:58584/solr

Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:58584/solr
	at __randomizedtesting.SeedInfo.seed([3B77B4AB27EAD1B:C24702E69F2E67BC]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
	at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestTlogReplica.testRecovery

Error Message:
Can not find doc 7 in https://127.0.0.1:58873/solr

Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:58873/solr
	at __randomizedtesting.SeedInfo.seed([3B77B4AB27EAD1B:C24702E69F2E67BC]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
	at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestTlogReplica.testRecovery

Error Message:
Can not find doc 7 in https://127.0.0.1:59906/solr

Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:59906/solr
	at __randomizedtesting.SeedInfo.seed([3B77B4AB27EAD1B:C24702E69F2E67BC]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:902)
	at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:567)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 13214 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestTlogReplica
   [junit4]   2> 1535027 INFO  (SUITE-TestTlogReplica-seed#[3B77B4AB27EAD1B]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_3B77B4AB27EAD1B-001/init-core-data-001
   [junit4]   2> 1535028 WARN  (SUITE-TestTlogReplica-seed#[3B77B4AB27EAD1B]-worker) [    ] o.a.s.SolrTestCaseJ4 startTrackingSearchers: numOpens=12 numCloses=12
   [junit4]   2> 1535028 INFO  (SUITE-TestTlogReplica-seed#[3B77B4AB27EAD1B]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 1535029 INFO  (SUITE-TestTlogReplica-seed#[3B77B4AB27EAD1B]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0) w/ MAC_OS_X supressed clientAuth
   [junit4]   2> 1535037 INFO  (SUITE-TestTlogReplica-seed#[3B77B4AB27EAD1B]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_3B77B4AB27EAD1B-001/tempDir-001
   [junit4]   2> 1535037 INFO  (SUITE-TestTlogReplica-seed#[3B77B4AB27EAD1B]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 1535045 INFO  (Thread-1972) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 1535045 INFO  (Thread-1972) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 1535049 ERROR (Thread-1972) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 1535152 INFO  (SUITE-TestTlogReplica-seed#[3B77B4AB27EAD1B]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:59904
   [junit4]   2> 1535178 INFO  (zkConnectionManagerCallback-3414-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1535191 INFO  (jetty-launcher-3411-thread-1) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 9+181
   [junit4]   2> 1535193 INFO  (jetty-launcher-3411-thread-2) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 9+181
   [junit4]   2> 1535198 INFO  (jetty-launcher-3411-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1535198 INFO  (jetty-launcher-3411-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1535199 INFO  (jetty-launcher-3411-thread-1) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1535199 INFO  (jetty-launcher-3411-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@4ffcd934{/solr,null,AVAILABLE}
   [junit4]   2> 1535204 INFO  (jetty-launcher-3411-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@7f685373{SSL,[ssl, http/1.1]}{127.0.0.1:59906}
   [junit4]   2> 1535205 INFO  (jetty-launcher-3411-thread-1) [    ] o.e.j.s.Server Started @1535277ms
   [junit4]   2> 1535205 INFO  (jetty-launcher-3411-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=59906}
   [junit4]   2> 1535205 ERROR (jetty-launcher-3411-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1535205 INFO  (jetty-launcher-3411-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1535207 INFO  (jetty-launcher-3411-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solrâ„¢ version 7.6.0
   [junit4]   2> 1535208 INFO  (jetty-launcher-3411-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1535208 INFO  (jetty-launcher-3411-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1535208 INFO  (jetty-launcher-3411-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-10-19T19:23:02.472082Z
   [junit4]   2> 1535210 INFO  (jetty-launcher-3411-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 1535210 INFO  (jetty-launcher-3411-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 1535210 INFO  (jetty-launcher-3411-thread-2) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 1535210 INFO  (jetty-launcher-3411-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@6c18df76{/solr,null,AVAILABLE}
   [junit4]   2> 1535211 INFO  (jetty-launcher-3411-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@3c354e89{SSL,[ssl, http/1.1]}{127.0.0.1:59907}
   [junit4]   2> 1535211 INFO  (jetty-launcher-3411-thread-2) [    ] o.e.j.s.Server Started @1535283ms
   [junit4]   2> 1535211 INFO  (jetty-launcher-3411-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=59907}
   [junit4]   2> 1535211 ERROR (jetty-launcher-3411-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 1535211 INFO  (jetty-launcher-3411-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 1535219 INFO  (jetty-launcher-3411-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solrâ„¢ version 7.6.0
   [junit4]   2> 1535220 INFO  (jetty-launcher-3411-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 1535220 INFO  (jetty-launcher-3411-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 1535220 INFO  (jetty-launcher-3411-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-10-19T19:23:02.484138Z
   [junit4]   2> 1535245 INFO  (zkConnectionManagerCallback-3418-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1535247 INFO  (jetty-launcher-3411-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1535265 INFO  (zkConnectionManagerCallback-3416-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1535270 INFO  (jetty-launcher-3411-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 1535473 INFO  (jetty-launcher-3411-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59904/solr
   [junit4]   2> 1535474 INFO  (zkConnectionManagerCallback-3422-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1535477 INFO  (zkConnectionManagerCallback-3424-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1535627 INFO  (jetty-launcher-3411-thread-2) [n:127.0.0.1:59907_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:59907_solr
   [junit4]   2> 1535628 INFO  (jetty-launcher-3411-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:59904/solr
   [junit4]   2> 1535628 INFO  (jetty-launcher-3411-thread-2) [n:127.0.0.1:59907_solr    ] o.a.s.c.Overseer Overseer (id=72205091761553412-127.0.0.1:59907_solr-n_0000000000) starting
   [junit4]   2> 1535630 INFO  (zkConnectionManagerCallback-3430-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1535633 INFO  (zkConnectionManagerCallback-3432-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1535639 INFO  (zkConnectionManagerCallback-3437-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1535642 INFO  (jetty-launcher-3411-thread-2) [n:127.0.0.1:59907_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59904/solr ready
   [junit4]   2> 1535644 INFO  (OverseerStateUpdate-72205091761553412-127.0.0.1:59907_solr-n_0000000000) [n:127.0.0.1:59907_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:59907_solr
   [junit4]   2> 1535645 INFO  (jetty-launcher-3411-thread-2) [n:127.0.0.1:59907_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59907_solr
   [junit4]   2> 1535648 INFO  (zkCallback-3423-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1535648 INFO  (zkCallback-3436-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1535676 INFO  (jetty-launcher-3411-thread-1) [n:127.0.0.1:59906_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 1535680 INFO  (jetty-launcher-3411-thread-2) [n:127.0.0.1:59907_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1535685 INFO  (jetty-launcher-3411-thread-1) [n:127.0.0.1:59906_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1535685 INFO  (jetty-launcher-3411-thread-1) [n:127.0.0.1:59906_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:59906_solr
   [junit4]   2> 1535686 INFO  (zkCallback-3423-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1535686 INFO  (zkCallback-3436-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1535686 INFO  (zkCallback-3431-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 1535698 INFO  (jetty-launcher-3411-thread-2) [n:127.0.0.1:59907_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59907.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@72c2a160
   [junit4]   2> 1535703 INFO  (zkConnectionManagerCallback-3445-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1535704 INFO  (jetty-launcher-3411-thread-1) [n:127.0.0.1:59906_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1535704 INFO  (jetty-launcher-3411-thread-2) [n:127.0.0.1:59907_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59907.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@72c2a160
   [junit4]   2> 1535705 INFO  (jetty-launcher-3411-thread-2) [n:127.0.0.1:59907_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59907.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@72c2a160
   [junit4]   2> 1535705 INFO  (jetty-launcher-3411-thread-1) [n:127.0.0.1:59906_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59904/solr ready
   [junit4]   2> 1535706 INFO  (jetty-launcher-3411-thread-2) [n:127.0.0.1:59907_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_3B77B4AB27EAD1B-001/tempDir-001/node2/.
   [junit4]   2> 1535706 INFO  (jetty-launcher-3411-thread-1) [n:127.0.0.1:59906_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 1535721 INFO  (jetty-launcher-3411-thread-1) [n:127.0.0.1:59906_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59906.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@72c2a160
   [junit4]   2> 1535735 INFO  (jetty-launcher-3411-thread-1) [n:127.0.0.1:59906_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59906.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@72c2a160
   [junit4]   2> 1535735 INFO  (jetty-launcher-3411-thread-1) [n:127.0.0.1:59906_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59906.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@72c2a160
   [junit4]   2> 1535736 INFO  (jetty-launcher-3411-thread-1) [n:127.0.0.1:59906_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_3B77B4AB27EAD1B-001/tempDir-001/node1/.
   [junit4]   2> 1535769 INFO  (zkConnectionManagerCallback-3448-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1535772 INFO  (zkConnectionManagerCallback-3453-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 1535774 INFO  (SUITE-TestTlogReplica-seed#[3B77B4AB27EAD1B]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 1535775 INFO  (SUITE-TestTlogReplica-seed#[3B77B4AB27EAD1B]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:59904/solr ready
   [junit4]   2> 1535781 INFO  (SUITE-TestTlogReplica-seed#[3B77B4AB27EAD1B]-worker) [    ] o.a.s.c.TestTlogReplica Using legacyCloud?: false
   [junit4]   2> 1535801 INFO  (qtp1849262540-11926) [n:127.0.0.1:59906_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :clusterprop with params val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1535802 INFO  (qtp1849262540-11926) [n:127.0.0.1:59906_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 1535825 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[3B77B4AB27EAD1B]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testKillTlogReplica
   [junit4]   2> 1535827 INFO  (qtp1849262540-11923) [n:127.0.0.1:59906_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&collection.configName=conf&maxShardsPerNode=100&name=tlog_replica_test_kill_tlog_replica&nrtReplicas=0&action=CREATE&numShards=1&tlogReplicas=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 1535830 INFO  (OverseerThreadFactory-4603-thread-1-processing-n:127.0.0.1:59907_solr) [n:127.0.0.1:59907_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection tlog_replica_test_kill_tlog_replica
   [junit4]   2> 1535944 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 1535959 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 1535967 INFO  (OverseerStateUpdate-72205091761553412-127.0.0.1:59907_solr-n_0000000000) [n:127.0.0.1:59907_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"tlog_replica_test_kill_tlog_replica",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"tlog_replica_test_kill_tlog_replica_shard1_replica_t1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:59907/solr",
   [junit4]   2>   "type":"TLOG",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1535971 INFO  (OverseerStateUpdate-72205091761553412-127.0.0.1:59907_solr-n_0000000000) [n:127.0.0.1:59907_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"tlog_replica_test_kill_tlog_replica",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"tlog_replica_test_kill_tlog_replica_shard1_replica_t2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:59906/solr",
   [junit4]   2>   "type":"TLOG",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 1536187 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr    x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=tlog_replica_test_kill_tlog_replica_shard1_replica_t2&action=CREATE&numShards=1&collection=tlog_replica_test_kill_tlog_replica&shard=shard1&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 1536187 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr    x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=tlog_replica_test_kill_tlog_replica_shard1_replica_t1&action=CREATE&numShards=1&collection=tlog_replica_test_kill_tlog_replica&shard=shard1&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 1536187 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr    x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 1537215 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.6.0
   [junit4]   2> 1537216 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.6.0
   [junit4]   2> 1537223 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.s.IndexSchema [tlog_replica_test_kill_tlog_replica_shard1_replica_t1] Schema name=minimal
   [junit4]   2> 1537240 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1537241 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_kill_tlog_replica_shard1_replica_t1' using configuration from collection tlog_replica_test_kill_tlog_replica, trusted=true
   [junit4]   2> 1537241 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59907.solr.core.tlog_replica_test_kill_tlog_replica.shard1.replica_t1' (registry 'solr.core.tlog_replica_test_kill_tlog_replica.shard1.replica_t1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@72c2a160
   [junit4]   2> 1537241 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.SolrCore [[tlog_replica_test_kill_tlog_replica_shard1_replica_t1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_3B77B4AB27EAD1B-001/tempDir-001/node2/tlog_replica_test_kill_tlog_replica_shard1_replica_t1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_3B77B4AB27EAD1B-001/tempDir-001/node2/./tlog_replica_test_kill_tlog_replica_shard1_replica_t1/data/]
   [junit4]   2> 1537243 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.s.IndexSchema [tlog_replica_test_kill_tlog_replica_shard1_replica_t2] Schema name=minimal
   [junit4]   2> 1537244 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 1537244 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_kill_tlog_replica_shard1_replica_t2' using configuration from collection tlog_replica_test_kill_tlog_replica, trusted=true
   [junit4]   2> 1537245 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_59906.solr.core.tlog_replica_test_kill_tlog_replica.shard1.replica_t2' (registry 'solr.core.tlog_replica_test_kill_tlog_replica.shard1.replica_t2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@72c2a160
   [junit4]   2> 1537245 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.SolrCore [[tlog_replica_test_kill_tlog_replica_shard1_replica_t2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_3B77B4AB27EAD1B-001/tempDir-001/node1/tlog_replica_test_kill_tlog_replica_shard1_replica_t2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_3B77B4AB27EAD1B-001/tempDir-001/node1/./tlog_replica_test_kill_tlog_replica_shard1_replica_t2/data/]
   [junit4]   2> 1537384 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1537384 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1537391 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 1537391 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 1537391 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1537391 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1537393 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 1537395 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.s.SolrIndexSearcher Opening [Searcher@5eef2e9[tlog_replica_test_kill_tlog_replica_shard1_replica_t2] main]
   [junit4]   2> 1537396 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 1537397 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 1537397 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 1537398 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1537398 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1614782906668941312
   [junit4]   2> 1537399 INFO  (searcherExecutor-4613-thread-1-processing-n:127.0.0.1:59906_solr x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2 c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.SolrCore [tlog_replica_test_kill_tlog_replica_shard1_replica_t2] Registered new searcher Searcher@5eef2e9[tlog_replica_test_kill_tlog_replica_shard1_replica_t2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1537401 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.s.SolrIndexSearcher Opening [Searcher@6f7822f3[tlog_replica_test_kill_tlog_replica_shard1_replica_t1] main]
   [junit4]   2> 1537403 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 1537403 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 1537404 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1537404 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1614782906675232768
   [junit4]   2> 1537408 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tlog_replica_test_kill_tlog_replica/terms/shard1 to Terms{values={core_node4=0}, version=0}
   [junit4]   2> 1537410 INFO  (searcherExecutor-4612-thread-1-processing-n:127.0.0.1:59907_solr x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1 c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.SolrCore [tlog_replica_test_kill_tlog_replica_shard1_replica_t1] Registered new searcher Searcher@6f7822f3[tlog_replica_test_kill_tlog_replica_shard1_replica_t1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1537416 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tlog_replica_test_kill_tlog_replica/terms/shard1 to Terms{values={core_node3=0, core_node4=0}, version=1}
   [junit4]   2> 1537418 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 1537418 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 1537418 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:59906/solr/tlog_replica_test_kill_tlog_replica_shard1_replica_t2/
   [junit4]   2> 1537419 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.PeerSync PeerSync: core=tlog_replica_test_kill_tlog_replica_shard1_replica_t2 url=https://127.0.0.1:59906/solr START replicas=[https://127.0.0.1:59907/solr/tlog_replica_test_kill_tlog_replica_shard1_replica_t1/] nUpdates=100
   [junit4]   2> 1537419 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.PeerSync PeerSync: core=tlog_replica_test_kill_tlog_replica_shard1_replica_t2 url=https://127.0.0.1:59906/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 1537437 INFO  (qtp876525391-11935) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=12
   [junit4]   2> 1537438 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 1537438 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 1537438 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 1537439 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.ZkController tlog_replica_test_kill_tlog_replica_shard1_replica_t2 stopping background replication from leader
   [junit4]   2> 1537446 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:59906/solr/tlog_replica_test_kill_tlog_replica_shard1_replica_t2/ shard1
   [junit4]   2> 1537559 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 1537564 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf&newCollection=true&name=tlog_replica_test_kill_tlog_replica_shard1_replica_t2&action=CREATE&numShards=1&collection=tlog_replica_test_kill_tlog_replica&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1377
   [junit4]   2> 1537673 INFO  (zkCallback-3431-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1538430 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.ZkController tlog_replica_test_kill_tlog_replica_shard1_replica_t1 starting background replication from leader
   [junit4]   2> 1538430 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
   [junit4]   2> 1538431 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
   [junit4]   2> 1538432 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 1538433 INFO  (indexFetcher-4623-thread-1) [    ] o.a.s.h.IndexFetcher Updated masterUrl to https://127.0.0.1:59906/solr/tlog_replica_test_kill_tlog_replica_shard1_replica_t2/
   [junit4]   2> 1538435 INFO  (qtp1849262540-11926) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 1538436 INFO  (indexFetcher-4623-thread-1) [    ] o.a.s.h.IndexFetcher Master's generation: 1
   [junit4]   2> 1538436 INFO  (indexFetcher-4623-thread-1) [    ] o.a.s.h.IndexFetcher Master's version: 0
   [junit4]   2> 1538436 INFO  (indexFetcher-4623-thread-1) [    ] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 1538436 INFO  (indexFetcher-4623-thread-1) [    ] o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 1538436 INFO  (indexFetcher-4623-thread-1) [    ] o.a.s.h.IndexFetcher New index in Master. Deleting mine...
   [junit4]   2> 1538437 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=tlog_replica_test_kill_tlog_replica_shard1_replica_t1&action=CREATE&numShards=1&collection=tlog_replica_test_kill_tlog_replica&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=2250
   [junit4]   2> 1538440 INFO  (indexFetcher-4623-thread-1) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@2455935c[tlog_replica_test_kill_tlog_replica_shard1_replica_t1] main]
   [junit4]   2> 1538442 INFO  (searcherExecutor-4612-thread-1) [    ] o.a.s.c.SolrCore [tlog_replica_test_kill_tlog_replica_shard1_replica_t1] Registered new searcher Searcher@2455935c[tlog_replica_test_kill_tlog_replica_shard1_replica_t1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 1538444 INFO  (qtp1849262540-11923) [n:127.0.0.1:59906_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 1538540 INFO  (zkCallback-3423-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1538540 INFO  (zkCallback-3431-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 1539449 INFO  (qtp1849262540-11923) [n:127.0.0.1:59906_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&collection.configName=conf&maxShardsPerNode=100&name=tlog_replica_test_kill_tlog_replica&nrtReplicas=0&action=CREATE&numShards=1&tlogReplicas=2&wt=javabin&version=2} status=0 QTime=3622
   [junit4]   2> 1539490 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1539495 INFO  (qtp1849262540-11923) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1539505 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tlog_replica_test_kill_tlog_replica/terms/shard1 to Terms{values={core_node3=1, core_node4=1}, version=2}
   [junit4]   2> 1539509 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:59906/solr/tlog_replica_test_kill_tlog_replica_shard1_replica_t2/&wt=javabin&version=2}{add=[1 (1614782908873048064)]} 0 1
   [junit4]   2> 1539510 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1 (1614782908873048064)]} 0 10
   [junit4]   2> 1539513 INFO  (qtp876525391-11934) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:59906/solr/tlog_replica_test_kill_tlog_replica_shard1_replica_t2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 0
   [junit4]   2> 1539518 INFO  (qtp1849262540-11926) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.DirectUpdateHandler2 start commit{_version_=1614782908891922432,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 1539518 INFO  (qtp1849262540-11926) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.SolrIndexWriter Calling setCommitData with IW:org.apache.solr.update.SolrIndexWriter@321a4f1 commitCommandVersion:1614782908891922432
   [junit4]   2> 1539543 INFO  (qtp1849262540-11926) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.s.SolrIndexSearcher Opening [Searcher@4503024c[tlog_replica_test_kill_tlog_replica_shard1_replica_t2] main]
   [junit4]   2> 1539544 INFO  (qtp1849262540-11926) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.DirectUpdateHandler2 end_commit_flush
   [junit4]   2> 1539546 INFO  (searcherExecutor-4613-thread-1-processing-n:127.0.0.1:59906_solr x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2 c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.SolrCore [tlog_replica_test_kill_tlog_replica_shard1_replica_t2] Registered new searcher Searcher@4503024c[tlog_replica_test_kill_tlog_replica_shard1_replica_t2] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.6.0):C1)))}
   [junit4]   2> 1539546 INFO  (qtp1849262540-11926) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&waitSearcher=true&openSearcher=true&commit=true&softCommit=false&distrib.from=https://127.0.0.1:59906/solr/tlog_replica_test_kill_tlog_replica_shard1_replica_t2/&commit_end_point=true&wt=javabin&version=2&expungeDeletes=false}{commit=} 0 29
   [junit4]   2> 1539547 INFO  (qtp1849262540-11923) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/update params={_stateVer_=tlog_replica_test_kill_tlog_replica:5&waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2}{commit=} 0 36
   [junit4]   2> 1539552 INFO  (qtp876525391-11934) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1539665 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1539766 INFO  (qtp876525391-11934) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1539849 INFO  (OverseerCollectionConfigSetProcessor-72205091761553412-127.0.0.1:59907_solr-n_0000000000) [n:127.0.0.1:59907_solr    ] o.a.s.c.OverseerTaskQueue Response ZK path: /overseer/collection-queue-work/qnr-0000000000 doesn't exist.  Requestor may have disconnected from ZooKeeper
   [junit4]   2> 1539870 INFO  (qtp876525391-11934) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1539986 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1540096 INFO  (qtp876525391-11934) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1540201 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1540308 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1540419 INFO  (qtp876525391-11934) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1540526 INFO  (qtp876525391-11934) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1540633 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1540742 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1540852 INFO  (qtp876525391-11934) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1540961 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1541068 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1541173 INFO  (qtp876525391-11934) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1541285 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1541387 INFO  (qtp876525391-11934) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1541435 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 1541436 INFO  (indexFetcher-4623-thread-1) [    ] o.a.s.h.IndexFetcher Master's generation: 2
   [junit4]   2> 1541436 INFO  (indexFetcher-4623-thread-1) [    ] o.a.s.h.IndexFetcher Master's version: 1539976986782
   [junit4]   2> 1541436 INFO  (indexFetcher-4623-thread-1) [    ] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 1541436 INFO  (indexFetcher-4623-thread-1) [    ] o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 1541436 INFO  (indexFetcher-4623-thread-1) [    ] o.a.s.h.IndexFetcher Starting replication process
   [junit4]   2> 1541437 INFO  (qtp1849262540-11923) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/replication params={generation=2&qt=/replication&tlogFiles=false&wt=javabin&version=2&command=filelist} status=0 QTime=1
   [junit4]   2> 1541438 INFO  (indexFetcher-4623-thread-1) [    ] o.a.s.h.IndexFetcher Number of files in latest index in master: 19
   [junit4]   2> 1541439 INFO  (indexFetcher-4623-thread-1) [    ] o.a.s.h.IndexFetcher Starting download (fullCopy=false) to MockDirectoryWrapper(RAMDirectory@3008d80f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3f415828)
   [junit4]   2> 1541440 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_BlockTreeOrds_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 1541445 INFO  (qtp1849262540-11923) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.si&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 1541445 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_FSTOrd50_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 1541450 INFO  (qtp1849262540-11923) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.dvdd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 1541452 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_FSTOrd50_0.pos&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 1541453 INFO  (qtp1849262540-11923) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 1541454 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdx&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 1541455 INFO  (qtp1849262540-11923) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_BlockTreeOrds_0.tio&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 1541456 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_BlockTreeOrds_0.doc&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 1541457 INFO  (qtp1849262540-11923) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fdt&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 1541458 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_BlockTreeOrds_0.tipo&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 1541459 INFO  (qtp1849262540-11923) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.dvd&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 1541461 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_FSTOrd50_0.tix&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 1541462 INFO  (qtp1849262540-11923) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Asserting_0.dvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 1541463 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_FSTOrd50_0.tbk&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 1541465 INFO  (qtp1849262540-11923) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.nvm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 1541477 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0_Direct_0.dvdm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 1541488 INFO  (qtp1849262540-11923) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=_0.fnm&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 1541491 INFO  (qtp876525391-11932) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=0 status=0 QTime=0
   [junit4]   2> 1541492 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/replication params={generation=2&qt=/replication&file=segments_2&checksum=true&wt=filestream&command=filecontent} status=0 QTime=0
   [junit4]   2> 1541492 INFO  (indexFetcher-4623-thread-1) [    ] o.a.s.h.IndexFetcher Bytes downloaded: 2939, Bytes skipped downloading: 0
   [junit4]   2> 1541492 INFO  (indexFetcher-4623-thread-1) [    ] o.a.s.h.IndexFetcher Total time taken for download (fullCopy=false,bytesDownloaded=2939) : 0 secs (null bytes/sec) to MockDirectoryWrapper(RAMDirectory@3008d80f lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@3f415828)
   [junit4]   2> 1541493 INFO  (indexFetcher-4623-thread-1) [    ] o.a.s.u.DefaultSolrCoreState New IndexWriter is ready to be used.
   [junit4]   2> 1541503 INFO  (indexFetcher-4623-thread-1) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@c9b4f02[tlog_replica_test_kill_tlog_replica_shard1_replica_t1] main]
   [junit4]   2> 1541504 INFO  (searcherExecutor-4612-thread-1) [    ] o.a.s.c.SolrCore [tlog_replica_test_kill_tlog_replica_shard1_replica_t1] Registered new searcher Searcher@c9b4f02[tlog_replica_test_kill_tlog_replica_shard1_replica_t1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(7.6.0):C1)))}
   [junit4]   2> 1541600 INFO  (qtp876525391-11934) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=0
   [junit4]   2> 1541604 INFO  (qtp1849262540-11925) [n:127.0.0.1:59906_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node4 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t2] o.a.s.c.S.Request [tlog_replica_test_kill_tlog_replica_shard1_replica_t2]  webapp=/solr path=/select params={q=*:*&wt=javabin&version=2} hits=1 status=0 QTime=0
   [junit4]   2> 1541605 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[3B77B4AB27EAD1B]) [    ] o.a.s.c.ChaosMonkey monkey: kill jetty! 59907
   [junit4]   2> 1541607 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[3B77B4AB27EAD1B]) [    ] o.e.j.s.AbstractConnector Stopped ServerConnector@3c354e89{SSL,[ssl, http/1.1]}{127.0.0.1:0}
   [junit4]   2> 1541607 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[3B77B4AB27EAD1B]) [    ] o.a.s.c.CoreContainer Shutting down CoreContainer instance=138876496
   [junit4]   2> 1541608 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[3B77B4AB27EAD1B]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.node, tag=null
   [junit4]   2> 1541608 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[3B77B4AB27EAD1B]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@464fa221: rootName = solr_59907, domain = solr.node, service url = null, agent id = null] for registry solr.node / com.codahale.metrics.MetricRegistry@504467
   [junit4]   2> 1541618 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[3B77B4AB27EAD1B]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jvm, tag=null
   [junit4]   2> 1541618 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[3B77B4AB27EAD1B]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@7a18b4b0: rootName = solr_59907, domain = solr.jvm, service url = null, agent id = null] for registry solr.jvm / com.codahale.metrics.MetricRegistry@441d60c9
   [junit4]   2> 1541625 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[3B77B4AB27EAD1B]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.jetty, tag=null
   [junit4]   2> 1541625 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[3B77B4AB27EAD1B]) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@32292d4b: rootName = solr_59907, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@6ecc83fd
   [junit4]   2> 1541627 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[3B77B4AB27EAD1B]) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:59907_solr
   [junit4]   2> 1541630 INFO  (zkCallback-3452-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 1541630 INFO  (zkCallback-3423-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 1541630 INFO  (zkCallback-3431-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 1541630 INFO  (zkCallback-3436-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 1541630 INFO  (zkCallback-3444-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 1541631 INFO  (zkCallback-3423-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1541631 INFO  (zkCallback-3431-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_kill_tlog_replica/state.json] for collection [tlog_replica_test_kill_tlog_replica] has occurred - updating... (live nodes size: [1])
   [junit4]   2> 1541632 INFO  (TEST-TestTlogReplica.testKillTlogReplica-seed#[3B77B4AB27EAD1B]) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 1541634 INFO  (coreCloseExecutor-4625-thread-1) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.c.SolrCore [tlog_replica_test_kill_tlog_replica_shard1_replica_t1]  CLOSING SolrCore org.apache.solr.core.SolrCore@7a77ebf8
   [junit4]   2> 1541634 INFO  (coreCloseExecutor-4625-thread-1) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.core.tlog_replica_test_kill_tlog_replica.shard1.replica_t1, tag=7a77ebf8
   [junit4]   2> 1541634 INFO  (coreCloseExecutor-4625-thread-1) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@5f351c82: rootName = solr_59907, domain = solr.core.tlog_replica_test_kill_tlog_replica.shard1.replica_t1, service url = null, agent id = null] for registry solr.core.tlog_replica_test_kill_tlog_replica.shard1.replica_t1 / com.codahale.metrics.MetricRegistry@2a3c13a7
   [junit4]   2> 1541644 INFO  (coreCloseExecutor-4625-thread-1) [n:127.0.0.1:59907_solr c:tlog_replica_test_kill_tlog_replica s:shard1 r:core_node3 x:tlog_replica_test_kill_tlog_replica_shard1_replica_t1] o.a.s.m.SolrMetricManager Cl

[...truncated too long message...]

O  (jetty-closer-266-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@2d2af58: rootName = solr_58873, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@2579ff81
   [junit4]   2> 348157 INFO  (jetty-closer-266-thread-2) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:58872_solr
   [junit4]   2> 348157 INFO  (jetty-closer-266-thread-1) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:58873_solr
   [junit4]   2> 348159 INFO  (jetty-closer-266-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 348159 INFO  (jetty-closer-266-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 348161 INFO  (zkCallback-392-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (0)
   [junit4]   2> 348161 INFO  (zkCallback-385-thread-2) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (0)
   [junit4]   2> 348162 INFO  (zkCallback-375-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (0)
   [junit4]   2> 348164 INFO  (zkCallback-368-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (0)
   [junit4]   2> 348165 INFO  (jetty-closer-266-thread-1) [    ] o.a.s.c.Overseer Overseer (id=72205324745834521-127.0.0.1:58873_solr-n_0000000005) closing
   [junit4]   2> 348166 INFO  (OverseerStateUpdate-72205324745834521-127.0.0.1:58873_solr-n_0000000005) [n:127.0.0.1:58873_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:58873_solr
   [junit4]   2> 348167 WARN  (OverseerAutoScalingTriggerThread-72205324745834521-127.0.0.1:58873_solr-n_0000000005) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 348171 INFO  (jetty-closer-266-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@9fd3349{/solr,null,UNAVAILABLE}
   [junit4]   2> 348171 INFO  (jetty-closer-266-thread-2) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 348173 INFO  (jetty-closer-266-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@12b40492{/solr,null,UNAVAILABLE}
   [junit4]   2> 348173 INFO  (jetty-closer-266-thread-1) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 348175 ERROR (SUITE-TestTlogReplica-seed#[3B77B4AB27EAD1B]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 348175 INFO  (SUITE-TestTlogReplica-seed#[3B77B4AB27EAD1B]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:58870 58870
   [junit4]   2> 348195 INFO  (Thread-153) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:58870 58870
   [junit4]   2> 348430 WARN  (Thread-153) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	32	/solr/configs/conf
   [junit4]   2> 	32	/solr/collections/tlog_replica_test_create_delete/terms/shard2
   [junit4]   2> 	28	/solr/collections/tlog_replica_test_create_delete/terms/shard1
   [junit4]   2> 	19	/solr/collections/tlog_replica_test_recovery/terms/shard1
   [junit4]   2> 	19	/solr/collections/tlog_replica_test_remove_leader/terms/shard1
   [junit4]   2> 	18	/solr/collections/tlog_replica_test_real_time_get/terms/shard1
   [junit4]   2> 	17	/solr/collections/tlog_replica_test_basic_leader_election/terms/shard1
   [junit4]   2> 	17	/solr/collections/tlog_replica_test_kill_tlog_replica/terms/shard1
   [junit4]   2> 	16	/solr/collections/tlog_replica_test_kill_leader/terms/shard1
   [junit4]   2> 	15	/solr/aliases.json
   [junit4]   2> 	14	/solr/collections/tlog_replica_test_out_of_order_db_qwith_in_place_updates/terms/shard1
   [junit4]   2> 	10	/solr/collections/tlog_replica_test_add_docs/terms/shard1
   [junit4]   2> 	9	/solr/collections/tlog_replica_test_delete_by_id/terms/shard1
   [junit4]   2> 	9	/solr/collections/tlog_replica_test_only_leader_indexes/terms/shard1
   [junit4]   2> 	8	/solr/collections/tlog_replica_test_add_remove_tlog_replica/terms/shard2
   [junit4]   2> 	7	/solr/security.json
   [junit4]   2> 	7	/solr/collections/tlog_replica_test_add_remove_tlog_replica/terms/shard1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	24	/solr/collections/tlog_replica_test_create_delete/state.json
   [junit4]   2> 	18	/solr/collections/tlog_replica_test_out_of_order_db_qwith_in_place_updates/state.json
   [junit4]   2> 	15	/solr/clusterstate.json
   [junit4]   2> 	15	/solr/clusterprops.json
   [junit4]   2> 	9	/solr/collections/tlog_replica_test_remove_leader/state.json
   [junit4]   2> 	8	/solr/collections/tlog_replica_test_kill_leader/state.json
   [junit4]   2> 	8	/solr/collections/tlog_replica_test_recovery/state.json
   [junit4]   2> 	8	/solr/collections/tlog_replica_test_basic_leader_election/state.json
   [junit4]   2> 	7	/solr/collections/tlog_replica_test_kill_tlog_replica/state.json
   [junit4]   2> 	7	/solr/collections/tlog_replica_test_real_time_get/state.json
   [junit4]   2> 	5	/solr/autoscaling.json
   [junit4]   2> 	3	/solr/collections/tlog_replica_test_add_docs/state.json
   [junit4]   2> 	3	/solr/collections/tlog_replica_test_create_delete/leader_elect/shard2/election/72205324745834502-core_node13-n_0000000001
   [junit4]   2> 	3	/solr/collections/tlog_replica_test_add_remove_tlog_replica/state.json
   [junit4]   2> 	3	/solr/collections/tlog_replica_test_only_leader_indexes/state.json
   [junit4]   2> 	3	/solr/collections/tlog_replica_test_delete_by_id/state.json
   [junit4]   2> 	2	/solr/overseer_elect/election/72205324745834513-127.0.0.1:58872_solr-n_0000000003
   [junit4]   2> 	2	/solr/collections/tlog_replica_test_real_time_get/leader_elect/shard1/election/72205324745834509-core_node3-n_0000000001
   [junit4]   2> 	2	/solr/collections/tlog_replica_test_create_delete/leader_elect/shard1/election/72205324745834502-core_node5-n_0000000001
   [junit4]   2> 	2	/solr/collections/tlog_replica_test_recovery/leader_elect/shard1/election/72205324745834513-core_node4-n_0000000000
   [junit4]   2> 	2	/solr/collections/tlog_replica_test_create_delete/leader_elect/shard2/election/72205324745834509-core_node11-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	15	/solr/live_nodes
   [junit4]   2> 	15	/solr/collections
   [junit4]   2> 	5	/solr/overseer/queue
   [junit4]   2> 	5	/solr/autoscaling/events/.scheduled_maintenance
   [junit4]   2> 	5	/solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 	5	/solr/overseer/collection-queue-work
   [junit4]   2> 	5	/solr/overseer/queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J0/temp/solr.cloud.TestTlogReplica_3B77B4AB27EAD1B-003
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {foo=BlockTreeOrds(blocksize=128), title_s=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=FSTOrd50, foo_s=BlockTreeOrds(blocksize=128)}, docValues:{_version_=DocValuesFormat(name=Asserting), id=DocValuesFormat(name=Direct), inplace_updatable_int=DocValuesFormat(name=Lucene70)}, maxPointsInLeafNode=811, maxMBSortInHeap=6.387630117523822, sim=RandomSimilarity(queryNorm=true): {}, locale=chr, timezone=America/Havana
   [junit4]   2> NOTE: Mac OS X 10.11.6 x86_64/Oracle Corporation 9 (64-bit)/cpus=3,threads=1,free=112056440,total=286085120
   [junit4]   2> NOTE: All tests run in this JVM: [TestTlogReplica, TestTlogReplica, TestTlogReplica]
   [junit4] Completed [5/5 (5!)] on J0 in 110.42s, 13 tests, 1 failure <<< FAILURES!

[...truncated 14 lines...]
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/common-build.xml:1567: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/common-build.xml:1092: There were test failures: 5 suites, 65 tests, 5 failures [seed: 3B77B4AB27EAD1B]

Total time: 5 minutes 52 seconds

[repro] Setting last failure code to 256

[repro] Failures:
[repro]   5/5 failed: org.apache.solr.cloud.TestTlogReplica
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=2f61f96bfae9d97e3536305e49865433e28737c2, workspace=/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX
[WARNINGS] Computing warning deltas based on reference build #887
Recording test results
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2

[JENKINS] Lucene-Solr-7.x-MacOSX (64bit/jdk-9) - Build # 887 - Still Unstable!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-MacOSX/887/
Java: 64bit/jdk-9 -XX:-UseCompressedOops -XX:+UseSerialGC

5 tests failed.
FAILED:  org.apache.solr.cloud.TestTlogReplica.testRecovery

Error Message:
Can not find doc 7 in https://127.0.0.1:50395/solr

Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:50395/solr
	at __randomizedtesting.SeedInfo.seed([968C6C528E719FD7:577C15FEA3215570]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:901)
	at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:566)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestTlogReplica.testRecovery

Error Message:
Can not find doc 7 in https://127.0.0.1:50740/solr

Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:50740/solr
	at __randomizedtesting.SeedInfo.seed([968C6C528E719FD7:577C15FEA3215570]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:901)
	at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:566)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestTlogReplica.testRecovery

Error Message:
Can not find doc 7 in https://127.0.0.1:50735/solr

Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:50735/solr
	at __randomizedtesting.SeedInfo.seed([968C6C528E719FD7:577C15FEA3215570]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:901)
	at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:566)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestTlogReplica.testRecovery

Error Message:
Can not find doc 7 in https://127.0.0.1:51071/solr

Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:51071/solr
	at __randomizedtesting.SeedInfo.seed([968C6C528E719FD7:577C15FEA3215570]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:901)
	at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:566)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)


FAILED:  org.apache.solr.cloud.TestTlogReplica.testRecovery

Error Message:
Can not find doc 7 in https://127.0.0.1:60029/solr

Stack Trace:
java.lang.AssertionError: Can not find doc 7 in https://127.0.0.1:60029/solr
	at __randomizedtesting.SeedInfo.seed([968C6C528E719FD7:577C15FEA3215570]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.junit.Assert.assertNotNull(Assert.java:526)
	at org.apache.solr.cloud.TestTlogReplica.checkRTG(TestTlogReplica.java:901)
	at org.apache.solr.cloud.TestTlogReplica.testRecovery(TestTlogReplica.java:566)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:564)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1742)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:935)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:971)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:985)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:944)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:830)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:880)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:891)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:57)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
	at java.base/java.lang.Thread.run(Thread.java:844)




Build Log:
[...truncated 14082 lines...]
   [junit4] Suite: org.apache.solr.cloud.TestTlogReplica
   [junit4]   2> Creating dataDir: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_968C6C528E719FD7-001/init-core-data-001
   [junit4]   2> 2567120 INFO  (SUITE-TestTlogReplica-seed#[968C6C528E719FD7]-worker) [    ] o.a.s.SolrTestCaseJ4 Using PointFields (NUMERIC_POINTS_SYSPROP=true) w/NUMERIC_DOCVALUES_SYSPROP=true
   [junit4]   2> 2567121 INFO  (SUITE-TestTlogReplica-seed#[968C6C528E719FD7]-worker) [    ] o.a.s.SolrTestCaseJ4 Randomized ssl (true) and clientAuth (false) via: @org.apache.solr.util.RandomizeSSL(reason="", value=0.0/0.0, ssl=0.0/0.0, clientAuth=0.0/0.0) w/ MAC_OS_X supressed clientAuth
   [junit4]   2> 2567122 INFO  (SUITE-TestTlogReplica-seed#[968C6C528E719FD7]-worker) [    ] o.a.s.SolrTestCaseJ4 SecureRandom sanity checks: test.solr.allowed.securerandom=null & java.security.egd=file:/dev/./urandom
   [junit4]   2> 2567122 INFO  (SUITE-TestTlogReplica-seed#[968C6C528E719FD7]-worker) [    ] o.a.s.c.MiniSolrCloudCluster Starting cluster of 2 servers in /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_968C6C528E719FD7-001/tempDir-001
   [junit4]   2> 2567122 INFO  (SUITE-TestTlogReplica-seed#[968C6C528E719FD7]-worker) [    ] o.a.s.c.ZkTestServer STARTING ZK TEST SERVER
   [junit4]   2> 2567122 INFO  (Thread-4098) [    ] o.a.s.c.ZkTestServer client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 2567122 INFO  (Thread-4098) [    ] o.a.s.c.ZkTestServer Starting server
   [junit4]   2> 2567135 ERROR (Thread-4098) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 2567228 INFO  (SUITE-TestTlogReplica-seed#[968C6C528E719FD7]-worker) [    ] o.a.s.c.ZkTestServer start zk server on port:60027
   [junit4]   2> 2567264 INFO  (zkConnectionManagerCallback-11062-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2567272 INFO  (jetty-launcher-11059-thread-2) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 9+181
   [junit4]   2> 2567272 INFO  (jetty-launcher-11059-thread-1) [    ] o.e.j.s.Server jetty-9.4.11.v20180605; built: 2018-06-05T18:24:03.829Z; git: d5fc0523cfa96bfebfbda19606cad384d772f04c; jvm 9+181
   [junit4]   2> 2567274 INFO  (jetty-launcher-11059-thread-1) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2567274 INFO  (jetty-launcher-11059-thread-1) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2567274 INFO  (jetty-launcher-11059-thread-1) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 2567274 INFO  (jetty-launcher-11059-thread-2) [    ] o.e.j.s.session DefaultSessionIdManager workerName=node0
   [junit4]   2> 2567274 INFO  (jetty-launcher-11059-thread-2) [    ] o.e.j.s.session No SessionScavenger set, using defaults
   [junit4]   2> 2567275 INFO  (jetty-launcher-11059-thread-2) [    ] o.e.j.s.session node0 Scavenging every 660000ms
   [junit4]   2> 2567275 INFO  (jetty-launcher-11059-thread-1) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@3fa5c16f{/solr,null,AVAILABLE}
   [junit4]   2> 2567275 INFO  (jetty-launcher-11059-thread-2) [    ] o.e.j.s.h.ContextHandler Started o.e.j.s.ServletContextHandler@628b830c{/solr,null,AVAILABLE}
   [junit4]   2> 2567275 INFO  (jetty-launcher-11059-thread-2) [    ] o.e.j.s.AbstractConnector Started ServerConnector@3e34c4c{SSL,[ssl, http/1.1]}{127.0.0.1:60030}
   [junit4]   2> 2567275 INFO  (jetty-launcher-11059-thread-1) [    ] o.e.j.s.AbstractConnector Started ServerConnector@394830b1{SSL,[ssl, http/1.1]}{127.0.0.1:60029}
   [junit4]   2> 2567275 INFO  (jetty-launcher-11059-thread-2) [    ] o.e.j.s.Server Started @2567350ms
   [junit4]   2> 2567275 INFO  (jetty-launcher-11059-thread-1) [    ] o.e.j.s.Server Started @2567350ms
   [junit4]   2> 2567275 INFO  (jetty-launcher-11059-thread-2) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=60030}
   [junit4]   2> 2567275 INFO  (jetty-launcher-11059-thread-1) [    ] o.a.s.c.s.e.JettySolrRunner Jetty properties: {hostContext=/solr, hostPort=60029}
   [junit4]   2> 2567276 ERROR (jetty-launcher-11059-thread-2) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2567276 ERROR (jetty-launcher-11059-thread-1) [    ] o.a.s.u.StartupLoggingUtils Missing Java Option solr.log.dir. Logging may be missing or incomplete.
   [junit4]   2> 2567276 INFO  (jetty-launcher-11059-thread-2) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 2567276 INFO  (jetty-launcher-11059-thread-1) [    ] o.a.s.s.SolrDispatchFilter Using logger factory org.apache.logging.slf4j.Log4jLoggerFactory
   [junit4]   2> 2567276 INFO  (jetty-launcher-11059-thread-1) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.6.0
   [junit4]   2> 2567276 INFO  (jetty-launcher-11059-thread-2) [    ] o.a.s.s.SolrDispatchFilter  ___      _       Welcome to Apache Solr? version 7.6.0
   [junit4]   2> 2567276 INFO  (jetty-launcher-11059-thread-1) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2567276 INFO  (jetty-launcher-11059-thread-2) [    ] o.a.s.s.SolrDispatchFilter / __| ___| |_ _   Starting in cloud mode on port null
   [junit4]   2> 2567276 INFO  (jetty-launcher-11059-thread-1) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2567276 INFO  (jetty-launcher-11059-thread-2) [    ] o.a.s.s.SolrDispatchFilter \__ \/ _ \ | '_|  Install dir: null
   [junit4]   2> 2567276 INFO  (jetty-launcher-11059-thread-1) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-10-16T14:44:24.316553Z
   [junit4]   2> 2567276 INFO  (jetty-launcher-11059-thread-2) [    ] o.a.s.s.SolrDispatchFilter |___/\___/_|_|    Start time: 2018-10-16T14:44:24.316561Z
   [junit4]   2> 2567278 INFO  (zkConnectionManagerCallback-11065-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2567278 INFO  (zkConnectionManagerCallback-11066-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2567279 INFO  (jetty-launcher-11059-thread-2) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2567279 INFO  (jetty-launcher-11059-thread-1) [    ] o.a.s.s.SolrDispatchFilter solr.xml found in ZooKeeper. Loading...
   [junit4]   2> 2567334 INFO  (jetty-launcher-11059-thread-1) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60027/solr
   [junit4]   2> 2567335 INFO  (zkConnectionManagerCallback-11070-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2567338 INFO  (zkConnectionManagerCallback-11072-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2567417 INFO  (jetty-launcher-11059-thread-2) [    ] o.a.s.c.ZkContainer Zookeeper client=127.0.0.1:60027/solr
   [junit4]   2> 2567418 INFO  (zkConnectionManagerCallback-11078-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2567421 INFO  (zkConnectionManagerCallback-11080-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2567440 INFO  (jetty-launcher-11059-thread-2) [n:127.0.0.1:60030_solr    ] o.a.s.c.OverseerElectionContext I am going to be the leader 127.0.0.1:60030_solr
   [junit4]   2> 2567442 INFO  (jetty-launcher-11059-thread-2) [n:127.0.0.1:60030_solr    ] o.a.s.c.Overseer Overseer (id=72187009188560902-127.0.0.1:60030_solr-n_0000000000) starting
   [junit4]   2> 2567452 INFO  (zkConnectionManagerCallback-11087-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2567455 INFO  (jetty-launcher-11059-thread-2) [n:127.0.0.1:60030_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:60027/solr ready
   [junit4]   2> 2567456 INFO  (OverseerStateUpdate-72187009188560902-127.0.0.1:60030_solr-n_0000000000) [n:127.0.0.1:60030_solr    ] o.a.s.c.Overseer Starting to work on the main queue : 127.0.0.1:60030_solr
   [junit4]   2> 2567456 INFO  (jetty-launcher-11059-thread-2) [n:127.0.0.1:60030_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60030_solr
   [junit4]   2> 2567457 INFO  (OverseerStateUpdate-72187009188560902-127.0.0.1:60030_solr-n_0000000000) [n:127.0.0.1:60030_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2567459 INFO  (zkCallback-11086-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2567475 INFO  (jetty-launcher-11059-thread-2) [n:127.0.0.1:60030_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 2567489 INFO  (jetty-launcher-11059-thread-1) [n:127.0.0.1:60029_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (1)
   [junit4]   2> 2567496 INFO  (jetty-launcher-11059-thread-2) [n:127.0.0.1:60030_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60030.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@299b7e9f
   [junit4]   2> 2567497 INFO  (jetty-launcher-11059-thread-1) [n:127.0.0.1:60029_solr    ] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2567497 INFO  (jetty-launcher-11059-thread-1) [n:127.0.0.1:60029_solr    ] o.a.s.c.ZkController Register node as live in ZooKeeper:/live_nodes/127.0.0.1:60029_solr
   [junit4]   2> 2567498 INFO  (zkCallback-11079-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2567498 INFO  (zkCallback-11086-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2567498 INFO  (zkCallback-11071-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (2)
   [junit4]   2> 2567506 INFO  (jetty-launcher-11059-thread-2) [n:127.0.0.1:60030_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60030.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@299b7e9f
   [junit4]   2> 2567507 INFO  (jetty-launcher-11059-thread-2) [n:127.0.0.1:60030_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60030.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@299b7e9f
   [junit4]   2> 2567508 INFO  (jetty-launcher-11059-thread-2) [n:127.0.0.1:60030_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_968C6C528E719FD7-001/tempDir-001/node2/.
   [junit4]   2> 2567518 INFO  (zkConnectionManagerCallback-11093-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2567520 INFO  (jetty-launcher-11059-thread-1) [n:127.0.0.1:60029_solr    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2567521 INFO  (jetty-launcher-11059-thread-1) [n:127.0.0.1:60029_solr    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:60027/solr ready
   [junit4]   2> 2567522 INFO  (jetty-launcher-11059-thread-1) [n:127.0.0.1:60029_solr    ] o.a.s.h.a.MetricsHistoryHandler No .system collection, keeping metrics history in memory.
   [junit4]   2> 2567545 INFO  (jetty-launcher-11059-thread-1) [n:127.0.0.1:60029_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60029.solr.node' (registry 'solr.node') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@299b7e9f
   [junit4]   2> 2567553 INFO  (jetty-launcher-11059-thread-1) [n:127.0.0.1:60029_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60029.solr.jvm' (registry 'solr.jvm') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@299b7e9f
   [junit4]   2> 2567553 INFO  (jetty-launcher-11059-thread-1) [n:127.0.0.1:60029_solr    ] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60029.solr.jetty' (registry 'solr.jetty') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@299b7e9f
   [junit4]   2> 2567554 INFO  (jetty-launcher-11059-thread-1) [n:127.0.0.1:60029_solr    ] o.a.s.c.CorePropertiesLocator Found 0 core definitions underneath /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_968C6C528E719FD7-001/tempDir-001/node1/.
   [junit4]   2> 2567573 INFO  (zkConnectionManagerCallback-11096-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2567577 INFO  (zkConnectionManagerCallback-11101-thread-1) [    ] o.a.s.c.c.ConnectionManager zkClient has connected
   [junit4]   2> 2567579 INFO  (SUITE-TestTlogReplica-seed#[968C6C528E719FD7]-worker) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (0) -> (2)
   [junit4]   2> 2567580 INFO  (SUITE-TestTlogReplica-seed#[968C6C528E719FD7]-worker) [    ] o.a.s.c.s.i.ZkClientClusterStateProvider Cluster at 127.0.0.1:60027/solr ready
   [junit4]   2> 2567586 INFO  (SUITE-TestTlogReplica-seed#[968C6C528E719FD7]-worker) [    ] o.a.s.c.TestTlogReplica Using legacyCloud?: false
   [junit4]   2> 2567600 INFO  (qtp1358363372-30110) [n:127.0.0.1:60030_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :clusterprop with params val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2567602 INFO  (qtp1358363372-30110) [n:127.0.0.1:60030_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={val=false&name=legacyCloud&action=CLUSTERPROP&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 2567623 INFO  (TEST-TestTlogReplica.testRealTimeGet-seed#[968C6C528E719FD7]) [    ] o.a.s.SolrTestCaseJ4 ###Starting testRealTimeGet
   [junit4]   2> 2567624 INFO  (qtp1358363372-30112) [n:127.0.0.1:60030_solr    ] o.a.s.h.a.CollectionsHandler Invoked Collection Action :create with params pullReplicas=0&collection.configName=conf&maxShardsPerNode=100&name=tlog_replica_test_real_time_get&nrtReplicas=2&action=CREATE&numShards=1&tlogReplicas=2&wt=javabin&version=2 and sendToOCPQueue=true
   [junit4]   2> 2567627 INFO  (OverseerThreadFactory-10481-thread-1-processing-n:127.0.0.1:60030_solr) [n:127.0.0.1:60030_solr    ] o.a.s.c.a.c.CreateCollectionCmd Create collection tlog_replica_test_real_time_get
   [junit4]   2> 2567734 WARN  (OverseerThreadFactory-10481-thread-1-processing-n:127.0.0.1:60030_solr) [n:127.0.0.1:60030_solr    ] o.a.s.c.a.c.CreateCollectionCmd Specified number of replicas of 4 on collection tlog_replica_test_real_time_get is higher than the number of Solr instances currently live or live and part of your createNodeSet(2). It's unusual to run two replica of the same slice on the same Solr-instance.
   [junit4]   2> 2567740 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 2567755 INFO  (qtp1916598279-30115) [n:127.0.0.1:60029_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/metrics params={prefix=CONTAINER.fs.usableSpace,CONTAINER.fs.totalSpace,CORE.coreName&wt=javabin&version=2&group=solr.node,solr.core} status=0 QTime=0
   [junit4]   2> 2567764 INFO  (OverseerStateUpdate-72187009188560902-127.0.0.1:60030_solr-n_0000000000) [n:127.0.0.1:60030_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"tlog_replica_test_real_time_get",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"tlog_replica_test_real_time_get_shard1_replica_n1",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:60029/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2567769 INFO  (OverseerStateUpdate-72187009188560902-127.0.0.1:60030_solr-n_0000000000) [n:127.0.0.1:60030_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"tlog_replica_test_real_time_get",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"tlog_replica_test_real_time_get_shard1_replica_n2",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:60030/solr",
   [junit4]   2>   "type":"NRT",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2567773 INFO  (OverseerStateUpdate-72187009188560902-127.0.0.1:60030_solr-n_0000000000) [n:127.0.0.1:60030_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"tlog_replica_test_real_time_get",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"tlog_replica_test_real_time_get_shard1_replica_t4",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:60030/solr",
   [junit4]   2>   "type":"TLOG",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2567777 INFO  (OverseerStateUpdate-72187009188560902-127.0.0.1:60030_solr-n_0000000000) [n:127.0.0.1:60030_solr    ] o.a.s.c.o.SliceMutator createReplica() {
   [junit4]   2>   "operation":"ADDREPLICA",
   [junit4]   2>   "collection":"tlog_replica_test_real_time_get",
   [junit4]   2>   "shard":"shard1",
   [junit4]   2>   "core":"tlog_replica_test_real_time_get_shard1_replica_t6",
   [junit4]   2>   "state":"down",
   [junit4]   2>   "base_url":"https://127.0.0.1:60029/solr",
   [junit4]   2>   "type":"TLOG",
   [junit4]   2>   "waitForFinalState":"false"} 
   [junit4]   2> 2567991 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr    x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf&newCollection=true&name=tlog_replica_test_real_time_get_shard1_replica_t4&action=CREATE&numShards=1&collection=tlog_replica_test_real_time_get&shard=shard1&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 2567991 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr    x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf&newCollection=true&name=tlog_replica_test_real_time_get_shard1_replica_t6&action=CREATE&numShards=1&collection=tlog_replica_test_real_time_get&shard=shard1&wt=javabin&version=2&replicaType=TLOG
   [junit4]   2> 2567991 INFO  (qtp1358363372-30110) [n:127.0.0.1:60030_solr    x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf&newCollection=true&name=tlog_replica_test_real_time_get_shard1_replica_n2&action=CREATE&numShards=1&collection=tlog_replica_test_real_time_get&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2567992 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr    x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.c.TransientSolrCoreCacheDefault Allocating transient cache for 2147483647 transient cores
   [junit4]   2> 2567991 INFO  (qtp1916598279-30118) [n:127.0.0.1:60029_solr    x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.h.a.CoreAdminOperation core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=tlog_replica_test_real_time_get_shard1_replica_n1&action=CREATE&numShards=1&collection=tlog_replica_test_real_time_get&shard=shard1&wt=javabin&version=2&replicaType=NRT
   [junit4]   2> 2569019 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.6.0
   [junit4]   2> 2569020 INFO  (qtp1358363372-30110) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.6.0
   [junit4]   2> 2569021 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.6.0
   [junit4]   2> 2569039 INFO  (qtp1916598279-30118) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.c.SolrConfig Using Lucene MatchVersion: 7.6.0
   [junit4]   2> 2569041 INFO  (qtp1358363372-30110) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.s.IndexSchema [tlog_replica_test_real_time_get_shard1_replica_n2] Schema name=minimal
   [junit4]   2> 2569041 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.s.IndexSchema [tlog_replica_test_real_time_get_shard1_replica_t6] Schema name=minimal
   [junit4]   2> 2569043 INFO  (qtp1358363372-30110) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2569043 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2569043 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_real_time_get_shard1_replica_t6' using configuration from collection tlog_replica_test_real_time_get, trusted=true
   [junit4]   2> 2569043 INFO  (qtp1358363372-30110) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_real_time_get_shard1_replica_n2' using configuration from collection tlog_replica_test_real_time_get, trusted=true
   [junit4]   2> 2569044 INFO  (qtp1358363372-30110) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60030.solr.core.tlog_replica_test_real_time_get.shard1.replica_n2' (registry 'solr.core.tlog_replica_test_real_time_get.shard1.replica_n2') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@299b7e9f
   [junit4]   2> 2569044 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60029.solr.core.tlog_replica_test_real_time_get.shard1.replica_t6' (registry 'solr.core.tlog_replica_test_real_time_get.shard1.replica_t6') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@299b7e9f
   [junit4]   2> 2569044 INFO  (qtp1358363372-30110) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.c.SolrCore [[tlog_replica_test_real_time_get_shard1_replica_n2] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_968C6C528E719FD7-001/tempDir-001/node2/tlog_replica_test_real_time_get_shard1_replica_n2], dataDir=[/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_968C6C528E719FD7-001/tempDir-001/node2/./tlog_replica_test_real_time_get_shard1_replica_n2/data/]
   [junit4]   2> 2569044 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.c.SolrCore [[tlog_replica_test_real_time_get_shard1_replica_t6] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_968C6C528E719FD7-001/tempDir-001/node1/tlog_replica_test_real_time_get_shard1_replica_t6], dataDir=[/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_968C6C528E719FD7-001/tempDir-001/node1/./tlog_replica_test_real_time_get_shard1_replica_t6/data/]
   [junit4]   2> 2569045 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.s.IndexSchema [tlog_replica_test_real_time_get_shard1_replica_t4] Schema name=minimal
   [junit4]   2> 2569046 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2569046 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_real_time_get_shard1_replica_t4' using configuration from collection tlog_replica_test_real_time_get, trusted=true
   [junit4]   2> 2569046 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60030.solr.core.tlog_replica_test_real_time_get.shard1.replica_t4' (registry 'solr.core.tlog_replica_test_real_time_get.shard1.replica_t4') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@299b7e9f
   [junit4]   2> 2569047 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.c.SolrCore [[tlog_replica_test_real_time_get_shard1_replica_t4] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_968C6C528E719FD7-001/tempDir-001/node2/tlog_replica_test_real_time_get_shard1_replica_t4], dataDir=[/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_968C6C528E719FD7-001/tempDir-001/node2/./tlog_replica_test_real_time_get_shard1_replica_t4/data/]
   [junit4]   2> 2569048 INFO  (qtp1916598279-30118) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.s.IndexSchema [tlog_replica_test_real_time_get_shard1_replica_n1] Schema name=minimal
   [junit4]   2> 2569049 INFO  (qtp1916598279-30118) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.s.IndexSchema Loaded schema minimal/1.1 with uniqueid field id
   [junit4]   2> 2569049 INFO  (qtp1916598279-30118) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.c.CoreContainer Creating SolrCore 'tlog_replica_test_real_time_get_shard1_replica_n1' using configuration from collection tlog_replica_test_real_time_get, trusted=true
   [junit4]   2> 2569049 INFO  (qtp1916598279-30118) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.m.r.SolrJmxReporter JMX monitoring for 'solr_60029.solr.core.tlog_replica_test_real_time_get.shard1.replica_n1' (registry 'solr.core.tlog_replica_test_real_time_get.shard1.replica_n1') enabled at server: com.sun.jmx.mbeanserver.JmxMBeanServer@299b7e9f
   [junit4]   2> 2569049 INFO  (qtp1916598279-30118) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.c.SolrCore [[tlog_replica_test_real_time_get_shard1_replica_n1] ] Opening new SolrCore at [/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_968C6C528E719FD7-001/tempDir-001/node1/tlog_replica_test_real_time_get_shard1_replica_n1], dataDir=[/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_968C6C528E719FD7-001/tempDir-001/node1/./tlog_replica_test_real_time_get_shard1_replica_n1/data/]
   [junit4]   2> 2569138 INFO  (qtp1358363372-30110) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2569138 INFO  (qtp1358363372-30110) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2569139 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2569139 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2569140 INFO  (qtp1358363372-30110) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2569140 INFO  (qtp1358363372-30110) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2569141 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2569141 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2569142 INFO  (qtp1358363372-30110) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.s.SolrIndexSearcher Opening [Searcher@3bc3608[tlog_replica_test_real_time_get_shard1_replica_n2] main]
   [junit4]   2> 2569143 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.s.SolrIndexSearcher Opening [Searcher@637dbf6b[tlog_replica_test_real_time_get_shard1_replica_t6] main]
   [junit4]   2> 2569144 INFO  (qtp1358363372-30110) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2569144 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2569144 INFO  (qtp1358363372-30110) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2569144 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2569145 INFO  (qtp1358363372-30110) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2569145 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2569145 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1614493585176002560
   [junit4]   2> 2569146 INFO  (searcherExecutor-10491-thread-1-processing-n:127.0.0.1:60030_solr x:tlog_replica_test_real_time_get_shard1_replica_n2 c:tlog_replica_test_real_time_get s:shard1 r:core_node5) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.c.SolrCore [tlog_replica_test_real_time_get_shard1_replica_n2] Registered new searcher Searcher@3bc3608[tlog_replica_test_real_time_get_shard1_replica_n2] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2569146 INFO  (qtp1358363372-30110) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1614493585177051136
   [junit4]   2> 2569146 INFO  (searcherExecutor-10490-thread-1-processing-n:127.0.0.1:60029_solr x:tlog_replica_test_real_time_get_shard1_replica_t6 c:tlog_replica_test_real_time_get s:shard1 r:core_node8) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.c.SolrCore [tlog_replica_test_real_time_get_shard1_replica_t6] Registered new searcher Searcher@637dbf6b[tlog_replica_test_real_time_get_shard1_replica_t6] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2569153 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tlog_replica_test_real_time_get/terms/shard1 to Terms{values={core_node8=0}, version=0}
   [junit4]   2> 2569163 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.c.ShardLeaderElectionContext Waiting until we see more replicas up for shard shard1: total=4 found=1 timeoutin=9998ms
   [junit4]   2> 2569162 INFO  (qtp1358363372-30110) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tlog_replica_test_real_time_get/terms/shard1 to Terms{values={core_node8=0, core_node5=0}, version=1}
   [junit4]   2> 2569207 INFO  (qtp1916598279-30118) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2569207 INFO  (qtp1916598279-30118) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2569208 INFO  (qtp1916598279-30118) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2569208 INFO  (qtp1916598279-30118) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2569210 INFO  (qtp1916598279-30118) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.s.SolrIndexSearcher Opening [Searcher@379f3e75[tlog_replica_test_real_time_get_shard1_replica_n1] main]
   [junit4]   2> 2569213 INFO  (qtp1916598279-30118) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2569214 INFO  (qtp1916598279-30118) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2569215 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.u.UpdateHandler Using UpdateLog implementation: org.apache.solr.update.UpdateLog
   [junit4]   2> 2569215 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.u.UpdateLog Initializing UpdateLog: dataDir=null defaultSyncLevel=FLUSH numRecordsToKeep=100 maxNumLogsToKeep=10 numVersionBuckets=65536
   [junit4]   2> 2569219 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.u.CommitTracker Hard AutoCommit: disabled
   [junit4]   2> 2569219 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.u.CommitTracker Soft AutoCommit: disabled
   [junit4]   2> 2569225 INFO  (qtp1916598279-30118) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2569225 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.s.SolrIndexSearcher Opening [Searcher@44b54302[tlog_replica_test_real_time_get_shard1_replica_t4] main]
   [junit4]   2> 2569251 INFO  (searcherExecutor-10493-thread-1-processing-n:127.0.0.1:60029_solr x:tlog_replica_test_real_time_get_shard1_replica_n1 c:tlog_replica_test_real_time_get s:shard1 r:core_node3) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.c.SolrCore [tlog_replica_test_real_time_get_shard1_replica_n1] Registered new searcher Searcher@379f3e75[tlog_replica_test_real_time_get_shard1_replica_n1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2569251 INFO  (qtp1916598279-30118) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1614493585287151616
   [junit4]   2> 2569252 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.r.ManagedResourceStorage Configured ZooKeeperStorageIO with znodeBase: /configs/conf
   [junit4]   2> 2569252 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.r.ManagedResourceStorage Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf
   [junit4]   2> 2569253 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2569253 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.u.UpdateLog Could not find max version in index or recent updates, using new clock 1614493585289248768
   [junit4]   2> 2569254 INFO  (searcherExecutor-10492-thread-1-processing-n:127.0.0.1:60030_solr x:tlog_replica_test_real_time_get_shard1_replica_t4 c:tlog_replica_test_real_time_get s:shard1 r:core_node7) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.c.SolrCore [tlog_replica_test_real_time_get_shard1_replica_t4] Registered new searcher Searcher@44b54302[tlog_replica_test_real_time_get_shard1_replica_t4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2569257 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tlog_replica_test_real_time_get/terms/shard1 to Terms{values={core_node7=0, core_node8=0, core_node5=0}, version=2}
   [junit4]   2> 2569257 INFO  (qtp1916598279-30118) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.c.ZkShardTerms Failed to save terms, version is not a match, retrying
   [junit4]   2> 2569260 INFO  (qtp1916598279-30118) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tlog_replica_test_real_time_get/terms/shard1 to Terms{values={core_node7=0, core_node8=0, core_node3=0, core_node5=0}, version=3}
   [junit4]   2> 2569675 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.c.ShardLeaderElectionContext Enough replicas found to continue.
   [junit4]   2> 2569675 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.c.ShardLeaderElectionContext I may be the new leader - try and sync
   [junit4]   2> 2569675 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.c.SyncStrategy Sync replicas to https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_t6/
   [junit4]   2> 2569676 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.u.PeerSync PeerSync: core=tlog_replica_test_real_time_get_shard1_replica_t6 url=https://127.0.0.1:60029/solr START replicas=[https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_n1/, https://127.0.0.1:60030/solr/tlog_replica_test_real_time_get_shard1_replica_n2/, https://127.0.0.1:60030/solr/tlog_replica_test_real_time_get_shard1_replica_t4/] nUpdates=100
   [junit4]   2> 2569676 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.u.PeerSync PeerSync: core=tlog_replica_test_real_time_get_shard1_replica_t6 url=https://127.0.0.1:60029/solr DONE.  We have no versions.  sync failed.
   [junit4]   2> 2569679 INFO  (qtp1916598279-30116) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.c.S.Request [tlog_replica_test_real_time_get_shard1_replica_n1]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2569679 INFO  (qtp1358363372-30122) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.c.S.Request [tlog_replica_test_real_time_get_shard1_replica_t4]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2569680 INFO  (qtp1358363372-30108) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.c.S.Request [tlog_replica_test_real_time_get_shard1_replica_n2]  webapp=/solr path=/get params={distrib=false&qt=/get&fingerprint=false&getVersions=100&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2569680 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.c.SyncStrategy Leader's attempt to sync with shard failed, moving to the next candidate
   [junit4]   2> 2569680 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.c.ShardLeaderElectionContext We failed sync, but we have no versions - we can't sync in that case - we were active before, so become leader anyway
   [junit4]   2> 2569680 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.c.ShardLeaderElectionContext Found all replicas participating in election, clear LIR
   [junit4]   2> 2569681 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.c.ZkController tlog_replica_test_real_time_get_shard1_replica_t6 stopping background replication from leader
   [junit4]   2> 2569690 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.c.ShardLeaderElectionContext I am the new leader: https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_t6/ shard1
   [junit4]   2> 2569797 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.c.ZkController I am the leader, no recovery necessary
   [junit4]   2> 2569799 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node8&collection.configName=conf&newCollection=true&name=tlog_replica_test_real_time_get_shard1_replica_t6&action=CREATE&numShards=1&collection=tlog_replica_test_real_time_get&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=1807
   [junit4]   2> 2569910 INFO  (zkCallback-11071-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_real_time_get/state.json] for collection [tlog_replica_test_real_time_get] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2570181 INFO  (qtp1358363372-30110) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node5&collection.configName=conf&newCollection=true&name=tlog_replica_test_real_time_get_shard1_replica_n2&action=CREATE&numShards=1&collection=tlog_replica_test_real_time_get&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2190
   [junit4]   2> 2570276 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.c.ZkController tlog_replica_test_real_time_get_shard1_replica_t4 starting background replication from leader
   [junit4]   2> 2570276 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.c.ReplicateFromLeader Will start replication from leader with poll interval: 00:00:03
   [junit4]   2> 2570277 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.h.ReplicationHandler Poll scheduled at an interval of 3000ms
   [junit4]   2> 2570277 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.h.ReplicationHandler Commits will be reserved for 10000ms.
   [junit4]   2> 2570279 INFO  (qtp1916598279-30118) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf&newCollection=true&name=tlog_replica_test_real_time_get_shard1_replica_n1&action=CREATE&numShards=1&collection=tlog_replica_test_real_time_get&shard=shard1&wt=javabin&version=2&replicaType=NRT} status=0 QTime=2287
   [junit4]   2> 2570280 INFO  (qtp1358363372-30121) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores params={qt=/admin/cores&coreNodeName=core_node7&collection.configName=conf&newCollection=true&name=tlog_replica_test_real_time_get_shard1_replica_t4&action=CREATE&numShards=1&collection=tlog_replica_test_real_time_get&shard=shard1&wt=javabin&version=2&replicaType=TLOG} status=0 QTime=2289
   [junit4]   2> 2570283 INFO  (qtp1358363372-30112) [n:127.0.0.1:60030_solr    ] o.a.s.h.a.CollectionsHandler Wait for new collection to be active for at most 30 seconds. Check all shard replicas
   [junit4]   2> 2570340 INFO  (indexFetcher-10511-thread-1) [    ] o.a.s.h.IndexFetcher Updated masterUrl to https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_t6/
   [junit4]   2> 2570341 INFO  (qtp1916598279-30115) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.c.S.Request [tlog_replica_test_real_time_get_shard1_replica_t6]  webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
   [junit4]   2> 2570341 INFO  (indexFetcher-10511-thread-1) [    ] o.a.s.h.IndexFetcher Master's generation: 1
   [junit4]   2> 2570341 INFO  (indexFetcher-10511-thread-1) [    ] o.a.s.h.IndexFetcher Master's version: 0
   [junit4]   2> 2570341 INFO  (indexFetcher-10511-thread-1) [    ] o.a.s.h.IndexFetcher Slave's generation: 1
   [junit4]   2> 2570341 INFO  (indexFetcher-10511-thread-1) [    ] o.a.s.h.IndexFetcher Slave's version: 0
   [junit4]   2> 2570341 INFO  (indexFetcher-10511-thread-1) [    ] o.a.s.h.IndexFetcher New index in Master. Deleting mine...
   [junit4]   2> 2570341 INFO  (indexFetcher-10511-thread-1) [    ] o.a.s.s.SolrIndexSearcher Opening [Searcher@7fc4546a[tlog_replica_test_real_time_get_shard1_replica_t4] main]
   [junit4]   2> 2570343 INFO  (searcherExecutor-10492-thread-1) [    ] o.a.s.c.SolrCore [tlog_replica_test_real_time_get_shard1_replica_t4] Registered new searcher Searcher@7fc4546a[tlog_replica_test_real_time_get_shard1_replica_t4] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 2570391 INFO  (zkCallback-11079-thread-1) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_real_time_get/state.json] for collection [tlog_replica_test_real_time_get] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2570391 INFO  (zkCallback-11071-thread-2) [    ] o.a.s.c.c.ZkStateReader A cluster state change: [WatchedEvent state:SyncConnected type:NodeDataChanged path:/collections/tlog_replica_test_real_time_get/state.json] for collection [tlog_replica_test_real_time_get] has occurred - updating... (live nodes size: [2])
   [junit4]   2> 2571284 INFO  (qtp1358363372-30112) [n:127.0.0.1:60030_solr    ] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/collections params={pullReplicas=0&collection.configName=conf&maxShardsPerNode=100&name=tlog_replica_test_real_time_get&nrtReplicas=2&action=CREATE&numShards=1&tlogReplicas=2&wt=javabin&version=2} status=0 QTime=3659
   [junit4]   2> 2571298 INFO  (qtp1916598279-30116) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.c.ZkShardTerms Successful update of terms at /collections/tlog_replica_test_real_time_get/terms/shard1 to Terms{values={core_node7=1, core_node8=1, core_node3=1, core_node5=1}, version=4}
   [junit4]   2> 2571301 INFO  (qtp1358363372-30106) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_real_time_get_shard1_replica_n2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_t6/&wt=javabin&version=2}{add=[0 (1614493587430440960)]} 0 1
   [junit4]   2> 2571301 INFO  (qtp1916598279-30114) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_real_time_get_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_t6/&wt=javabin&version=2}{add=[0 (1614493587430440960)]} 0 1
   [junit4]   2> 2571301 INFO  (qtp1358363372-30108) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_real_time_get_shard1_replica_t4]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_t6/&wt=javabin&version=2}{add=[0 (1614493587430440960)]} 0 1
   [junit4]   2> 2571301 INFO  (qtp1916598279-30116) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_real_time_get_shard1_replica_t6]  webapp=/solr path=/update params={update.distrib=TOLEADER&distrib.from=https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_n1/&wt=javabin&version=2}{add=[0 (1614493587430440960)]} 0 6
   [junit4]   2> 2571301 INFO  (qtp1916598279-30115) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_real_time_get_shard1_replica_n1]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[0]} 0 8
   [junit4]   2> 2571306 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.c.S.Request [tlog_replica_test_real_time_get_shard1_replica_n1]  webapp=/solr path=/get params={qt=/get&_stateVer_=tlog_replica_test_real_time_get:6&ids=0&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2571306 INFO  (qtp1916598279-30116) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.c.S.Request [tlog_replica_test_real_time_get_shard1_replica_n1]  webapp=/solr path=/get params={qt=/get&ids=0&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2571307 INFO  (qtp1358363372-30122) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.c.S.Request [tlog_replica_test_real_time_get_shard1_replica_n2]  webapp=/solr path=/get params={qt=/get&ids=0&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2571308 INFO  (qtp1358363372-30108) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.h.c.RealTimeGetComponent LOOKUP_SLICE:shard1=https://127.0.0.1:60030/solr/tlog_replica_test_real_time_get_shard1_replica_n2/|https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_t6/|https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_n1/
   [junit4]   2> 2571308 INFO  (qtp1358363372-30122) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.c.S.Request [tlog_replica_test_real_time_get_shard1_replica_n2]  webapp=/solr path=/get params={distrib=false&qt=/get&omitHeader=true&shards.purpose=1&NOW=1539701068347&ids=0&isShard=true&shard.url=https://127.0.0.1:60030/solr/tlog_replica_test_real_time_get_shard1_replica_n2/|https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_t6/|https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_n1/&wt=javabin&version=2&shards.qt=/get} status=0 QTime=0
   [junit4]   2> 2571309 INFO  (qtp1358363372-30108) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.c.S.Request [tlog_replica_test_real_time_get_shard1_replica_t4]  webapp=/solr path=/get params={qt=/get&ids=0&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 2571309 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.h.c.RealTimeGetComponent LOOKUP_SLICE:shard1=https://127.0.0.1:60030/solr/tlog_replica_test_real_time_get_shard1_replica_n2/|https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_t6/|https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_n1/
   [junit4]   2> 2571312 INFO  (qtp1358363372-30106) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.c.S.Request [tlog_replica_test_real_time_get_shard1_replica_n2]  webapp=/solr path=/get params={distrib=false&qt=/get&omitHeader=true&shards.purpose=1&NOW=1539701068349&ids=0&isShard=true&shard.url=https://127.0.0.1:60030/solr/tlog_replica_test_real_time_get_shard1_replica_n2/|https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_t6/|https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_n1/&wt=javabin&version=2&shards.qt=/get} status=0 QTime=0
   [junit4]   2> 2571312 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.c.S.Request [tlog_replica_test_real_time_get_shard1_replica_t6]  webapp=/solr path=/get params={qt=/get&ids=0&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 2571316 INFO  (qtp1358363372-30106) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_real_time_get_shard1_replica_n2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_t6/&wt=javabin&version=2}{add=[1 (1614493587451412480)]} 0 0
   [junit4]   2> 2571317 INFO  (qtp1916598279-30116) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_real_time_get_shard1_replica_n1]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_t6/&wt=javabin&version=2}{add=[1 (1614493587451412480)]} 0 1
   [junit4]   2> 2571318 INFO  (qtp1358363372-30122) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_real_time_get_shard1_replica_t4]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_t6/&wt=javabin&version=2}{add=[1 (1614493587451412480)]} 0 1
   [junit4]   2> 2571318 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_real_time_get_shard1_replica_t6]  webapp=/solr path=/update params={update.distrib=TOLEADER&distrib.from=https://127.0.0.1:60030/solr/tlog_replica_test_real_time_get_shard1_replica_n2/&wt=javabin&version=2}{add=[1 (1614493587451412480)]} 0 3
   [junit4]   2> 2571318 INFO  (qtp1358363372-30108) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_real_time_get_shard1_replica_n2]  webapp=/solr path=/update params={wt=javabin&version=2}{add=[1]} 0 5
   [junit4]   2> 2571319 INFO  (qtp1358363372-30106) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.c.S.Request [tlog_replica_test_real_time_get_shard1_replica_n2]  webapp=/solr path=/get params={qt=/get&_stateVer_=tlog_replica_test_real_time_get:6&ids=1&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2571320 INFO  (qtp1916598279-30114) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.c.S.Request [tlog_replica_test_real_time_get_shard1_replica_n1]  webapp=/solr path=/get params={qt=/get&ids=1&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2571320 INFO  (qtp1358363372-30122) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.c.S.Request [tlog_replica_test_real_time_get_shard1_replica_n2]  webapp=/solr path=/get params={qt=/get&ids=1&wt=javabin&version=2} status=0 QTime=0
   [junit4]   2> 2571321 INFO  (qtp1358363372-30106) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.h.c.RealTimeGetComponent LOOKUP_SLICE:shard1=https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_n1/|https://127.0.0.1:60030/solr/tlog_replica_test_real_time_get_shard1_replica_n2/|https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_t6/
   [junit4]   2> 2571322 INFO  (qtp1916598279-30117) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.c.S.Request [tlog_replica_test_real_time_get_shard1_replica_n1]  webapp=/solr path=/get params={distrib=false&qt=/get&omitHeader=true&shards.purpose=1&NOW=1539701068360&ids=1&isShard=true&shard.url=https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_n1/|https://127.0.0.1:60030/solr/tlog_replica_test_real_time_get_shard1_replica_n2/|https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_t6/&wt=javabin&version=2&shards.qt=/get} status=0 QTime=0
   [junit4]   2> 2571322 INFO  (qtp1358363372-30106) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node7 x:tlog_replica_test_real_time_get_shard1_replica_t4] o.a.s.c.S.Request [tlog_replica_test_real_time_get_shard1_replica_t4]  webapp=/solr path=/get params={qt=/get&ids=1&wt=javabin&version=2} status=0 QTime=1
   [junit4]   2> 2571323 INFO  (qtp1916598279-30114) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.h.c.RealTimeGetComponent LOOKUP_SLICE:shard1=https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_n1/|https://127.0.0.1:60030/solr/tlog_replica_test_real_time_get_shard1_replica_n2/|https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_t6/
   [junit4]   2> 2571325 INFO  (qtp1916598279-30116) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node3 x:tlog_replica_test_real_time_get_shard1_replica_n1] o.a.s.c.S.Request [tlog_replica_test_real_time_get_shard1_replica_n1]  webapp=/solr path=/get params={distrib=false&qt=/get&omitHeader=true&shards.purpose=1&NOW=1539701068363&ids=1&isShard=true&shard.url=https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_n1/|https://127.0.0.1:60030/solr/tlog_replica_test_real_time_get_shard1_replica_n2/|https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_t6/&wt=javabin&version=2&shards.qt=/get} status=0 QTime=0
   [junit4]   2> 2571325 INFO  (qtp1916598279-30114) [n:127.0.0.1:60029_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node8 x:tlog_replica_test_real_time_get_shard1_replica_t6] o.a.s.c.S.Request [tlog_replica_test_real_time_get_shard1_replica_t6]  webapp=/solr path=/get params={qt=/get&ids=1&wt=javabin&version=2} status=0 QTime=2
   [junit4]   2> 2571328 INFO  (qtp1358363372-30106) [n:127.0.0.1:60030_solr c:tlog_replica_test_real_time_get s:shard1 r:core_node5 x:tlog_replica_test_real_time_get_shard1_replica_n2] o.a.s.u.p.LogUpdateProcessorFactory [tlog_replica_test_real_time_get_shard1_replica_n2]  webapp=/solr path=/update params={update.distrib=FROMLEADER&distrib.from=https://127.0.0.1:60029/solr/tlog_replica_test_real_time_get_shard1_replica_t6/&wt=javabin&version=2}{add=[2 (1614493587463995392)]} 0 0
   [junit4]   2> 2571329 INFO  (qtp1916598279-30114) [n:127.0.0.1:60029_sol

[...truncated too long message...]

2> 345407 INFO  (jetty-closer-266-thread-1) [    ] o.a.s.m.r.SolrJmxReporter Closing reporter [org.apache.solr.metrics.reporters.SolrJmxReporter@ded9398: rootName = solr_51072, domain = solr.jetty, service url = null, agent id = null] for registry solr.jetty / com.codahale.metrics.MetricRegistry@6197c8e5
   [junit4]   2> 345408 INFO  (jetty-closer-266-thread-2) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:51071_solr
   [junit4]   2> 345410 INFO  (jetty-closer-266-thread-2) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 345412 INFO  (zkCallback-314-thread-5) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 345412 INFO  (zkCallback-321-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 345413 INFO  (zkCallback-384-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 345413 INFO  (zkCallback-391-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (2) -> (1)
   [junit4]   2> 345415 INFO  (jetty-closer-266-thread-1) [    ] o.a.s.c.ZkController Remove node as live in ZooKeeper:/live_nodes/127.0.0.1:51072_solr
   [junit4]   2> 345418 INFO  (jetty-closer-266-thread-1) [    ] o.a.s.m.SolrMetricManager Closing metric reporters for registry=solr.cluster, tag=null
   [junit4]   2> 345420 INFO  (jetty-closer-266-thread-2) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@6d85b18f{/solr,null,UNAVAILABLE}
   [junit4]   2> 345420 INFO  (zkCallback-321-thread-1) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 345420 INFO  (zkCallback-314-thread-5) [    ] o.a.s.c.c.ZkStateReader Updated live nodes from ZooKeeper... (1) -> (0)
   [junit4]   2> 345420 INFO  (jetty-closer-266-thread-2) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 345424 INFO  (jetty-closer-266-thread-1) [    ] o.a.s.c.Overseer Overseer (id=72187157904490509-127.0.0.1:51072_solr-n_0000000002) closing
   [junit4]   2> 345424 INFO  (OverseerStateUpdate-72187157904490509-127.0.0.1:51072_solr-n_0000000002) [n:127.0.0.1:51072_solr    ] o.a.s.c.Overseer Overseer Loop exiting : 127.0.0.1:51072_solr
   [junit4]   2> 345425 WARN  (OverseerAutoScalingTriggerThread-72187157904490509-127.0.0.1:51072_solr-n_0000000002) [    ] o.a.s.c.a.OverseerTriggerThread OverseerTriggerThread woken up but we are closed, exiting.
   [junit4]   2> 345428 INFO  (jetty-closer-266-thread-1) [    ] o.e.j.s.h.ContextHandler Stopped o.e.j.s.ServletContextHandler@19dbc48e{/solr,null,UNAVAILABLE}
   [junit4]   2> 345428 INFO  (jetty-closer-266-thread-1) [    ] o.e.j.s.session node0 Stopped scavenging
   [junit4]   2> 345429 ERROR (SUITE-TestTlogReplica-seed#[968C6C528E719FD7]-worker) [    ] o.a.z.s.ZooKeeperServer ZKShutdownHandler is not registered, so ZooKeeper server won't take any action on ERROR or SHUTDOWN server state changes
   [junit4]   2> 345429 INFO  (SUITE-TestTlogReplica-seed#[968C6C528E719FD7]-worker) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:51069 51069
   [junit4]   2> 345444 INFO  (Thread-155) [    ] o.a.s.c.ZkTestServer connecting to 127.0.0.1:51069 51069
   [junit4]   2> 345703 WARN  (Thread-155) [    ] o.a.s.c.ZkTestServer Watch limit violations: 
   [junit4]   2> Maximum concurrent create/delete watches above limit:
   [junit4]   2> 
   [junit4]   2> 	33	/solr/collections/tlog_replica_test_create_delete/terms/shard1
   [junit4]   2> 	32	/solr/configs/conf
   [junit4]   2> 	31	/solr/collections/tlog_replica_test_create_delete/terms/shard2
   [junit4]   2> 	19	/solr/collections/tlog_replica_test_remove_leader/terms/shard1
   [junit4]   2> 	18	/solr/collections/tlog_replica_test_recovery/terms/shard1
   [junit4]   2> 	18	/solr/collections/tlog_replica_test_real_time_get/terms/shard1
   [junit4]   2> 	17	/solr/collections/tlog_replica_test_kill_leader/terms/shard1
   [junit4]   2> 	16	/solr/collections/tlog_replica_test_basic_leader_election/terms/shard1
   [junit4]   2> 	16	/solr/collections/tlog_replica_test_kill_tlog_replica/terms/shard1
   [junit4]   2> 	15	/solr/aliases.json
   [junit4]   2> 	14	/solr/collections/tlog_replica_test_out_of_order_db_qwith_in_place_updates/terms/shard1
   [junit4]   2> 	11	/solr/collections/tlog_replica_test_add_docs/terms/shard1
   [junit4]   2> 	11	/solr/collections/tlog_replica_test_delete_by_id/terms/shard1
   [junit4]   2> 	11	/solr/collections/tlog_replica_test_only_leader_indexes/terms/shard1
   [junit4]   2> 	8	/solr/collections/tlog_replica_test_add_remove_tlog_replica/terms/shard2
   [junit4]   2> 	7	/solr/security.json
   [junit4]   2> 	7	/solr/collections/tlog_replica_test_add_remove_tlog_replica/terms/shard1
   [junit4]   2> 
   [junit4]   2> Maximum concurrent data watches above limit:
   [junit4]   2> 
   [junit4]   2> 	23	/solr/collections/tlog_replica_test_create_delete/state.json
   [junit4]   2> 	15	/solr/collections/tlog_replica_test_out_of_order_db_qwith_in_place_updates/state.json
   [junit4]   2> 	15	/solr/clusterstate.json
   [junit4]   2> 	15	/solr/clusterprops.json
   [junit4]   2> 	11	/solr/collections/tlog_replica_test_kill_leader/state.json
   [junit4]   2> 	11	/solr/collections/tlog_replica_test_recovery/state.json
   [junit4]   2> 	11	/solr/collections/tlog_replica_test_remove_leader/state.json
   [junit4]   2> 	8	/solr/collections/tlog_replica_test_basic_leader_election/state.json
   [junit4]   2> 	7	/solr/collections/tlog_replica_test_kill_tlog_replica/state.json
   [junit4]   2> 	5	/solr/collections/tlog_replica_test_real_time_get/state.json
   [junit4]   2> 	4	/solr/collections/tlog_replica_test_add_docs/state.json
   [junit4]   2> 	4	/solr/overseer_elect/election/72187157904490509-127.0.0.1:51072_solr-n_0000000002
   [junit4]   2> 	3	/solr/autoscaling.json
   [junit4]   2> 	3	/solr/collections/tlog_replica_test_create_delete/leader_elect/shard1/election/72187157904490509-core_node7-n_0000000000
   [junit4]   2> 	3	/solr/collections/tlog_replica_test_add_remove_tlog_replica/state.json
   [junit4]   2> 	3	/solr/collections/tlog_replica_test_only_leader_indexes/state.json
   [junit4]   2> 	3	/solr/collections/tlog_replica_test_delete_by_id/state.json
   [junit4]   2> 	2	/solr/collections/tlog_replica_test_kill_tlog_replica/leader_elect/shard1/election/72187157904490509-core_node4-n_0000000000
   [junit4]   2> 	2	/solr/collections/tlog_replica_test_recovery/leader_elect/shard1/election/72187157904490509-core_node4-n_0000000000
   [junit4]   2> 	2	/solr/collections/tlog_replica_test_create_delete/leader_elect/shard1/election/72187157904490517-core_node3-n_0000000000
   [junit4]   2> 
   [junit4]   2> Maximum concurrent children watches above limit:
   [junit4]   2> 
   [junit4]   2> 	15	/solr/live_nodes
   [junit4]   2> 	15	/solr/collections
   [junit4]   2> 	3	/solr/overseer/queue
   [junit4]   2> 	3	/solr/autoscaling/events/.scheduled_maintenance
   [junit4]   2> 	3	/solr/autoscaling/events/.auto_add_replicas
   [junit4]   2> 	3	/solr/overseer/collection-queue-work
   [junit4]   2> 	3	/solr/overseer/queue-work
   [junit4]   2> 
   [junit4]   2> NOTE: leaving temporary files on disk at: /Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/solr/build/solr-core/test/J1/temp/solr.cloud.TestTlogReplica_968C6C528E719FD7-003
   [junit4]   2> Oct 16, 2018 3:23:59 PM com.carrotsearch.randomizedtesting.ThreadLeakControl checkThreadLeaks
   [junit4]   2> WARNING: Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> NOTE: test params are: codec=Lucene70, sim=RandomSimilarity(queryNorm=true): {}, locale=ee, timezone=America/Indiana/Tell_City
   [junit4]   2> NOTE: Mac OS X 10.11.6 x86_64/Oracle Corporation 9 (64-bit)/cpus=3,threads=1,free=57819608,total=319905792
   [junit4]   2> NOTE: All tests run in this JVM: [TestTlogReplica, TestTlogReplica, TestTlogReplica]
   [junit4] Completed [5/5 (5!)] on J1 in 106.40s, 13 tests, 1 failure <<< FAILURES!

[...truncated 14 lines...]
BUILD FAILED
/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/common-build.xml:1567: The following error occurred while executing this line:
/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX/lucene/common-build.xml:1092: There were test failures: 5 suites, 65 tests, 5 failures [seed: 968C6C528E719FD7]

Total time: 5 minutes 50 seconds

[repro] Setting last failure code to 256

[repro] Failures:
[repro]   5/5 failed: org.apache.solr.cloud.TestTlogReplica
[repro] Exiting with code 256
+ mv lucene/build lucene/build.repro
+ mv solr/build solr/build.repro
+ mv lucene/build.orig lucene/build
+ mv solr/build.orig solr/build
Archiving artifacts
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
[WARNINGS] Parsing warnings in console log with parser Java Compiler (javac)
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
<Git Blamer> Using GitBlamer to create author and commit information for all warnings.
<Git Blamer> GIT_COMMIT=7a2504e18c1508ed5b44ef5692d216b19f9d6a6b, workspace=/Users/jenkins/workspace/Lucene-Solr-7.x-MacOSX
[WARNINGS] Computing warning deltas based on reference build #886
Recording test results
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Build step 'Publish JUnit test result report' changed build result to UNSTABLE
Email was triggered for: Unstable (Test Failures)
Sending email for trigger: Unstable (Test Failures)
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2
Setting ANT_1_8_2_HOME=/Users/jenkins/tools/hudson.tasks.Ant_AntInstallation/ANT_1.8.2