You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2014/11/29 21:31:16 UTC

[JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.8.0_40-ea-b09) - Build # 4460 - Still Failing!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4460/
Java: 64bit/jdk1.8.0_40-ea-b09 -XX:+UseCompressedOops -XX:+UseG1GC (asserts: true)

1 tests failed.
FAILED:  org.apache.lucene.util.junitcompat.TestLeaveFilesIfTestFails.testWindowsUnremovableFile

Error Message:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\core\test\J0\temp\lucene.util.junitcompat.TestLeaveFilesIfTestFails$Nested2-C5F2CA80130DA348-001\leftover-001\child.locked

Stack Trace:
java.nio.file.AccessDeniedException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\core\test\J0\temp\lucene.util.junitcompat.TestLeaveFilesIfTestFails$Nested2-C5F2CA80130DA348-001\leftover-001\child.locked
	at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
	at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
	at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
	at org.apache.lucene.mockfile.WindowsFS.getKey(WindowsFS.java:57)
	at org.apache.lucene.mockfile.WindowsFS.onClose(WindowsFS.java:77)
	at org.apache.lucene.mockfile.HandleTrackingFS$5.close(HandleTrackingFS.java:210)
	at org.apache.lucene.util.junitcompat.TestLeaveFilesIfTestFails.testWindowsUnremovableFile(TestLeaveFilesIfTestFails.java:79)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.junit.rules.RunRules.evaluate(RunRules.java:18)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.junit.rules.RunRules.evaluate(RunRules.java:18)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
	at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:199)
	at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:310)
	at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)




Build Log:
[...truncated 436 lines...]
   [junit4] Suite: org.apache.lucene.util.junitcompat.TestLeaveFilesIfTestFails
   [junit4]   1> sysout from nested test: 
   [junit4]   1> 
   [junit4]   1> syserr from nested test: NOTE: test params are: codec=Asserting(Lucene50): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=sr_CS, timezone=Etc/UCT
   [junit4]   1> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.8.0_40-ea (64-bit)/cpus=2,threads=1,free=81523472,total=214958080
   [junit4]   1> NOTE: All tests run in this JVM: [TestAutomaton, TestQueryBuilder, Test2BNumericDocValues, TestDirectory, TestDocumentsWriterDeleteQueue, TestBooleanOr, TestPositiveScoresOnlyCollector, TestMmapDirectory, TestBagOfPostings, TestOperations, TestFieldReuse, TestBooleanScorer, TestNoMergeScheduler, TestFieldMaskingSpanQuery, TestHighCompressionMode, TestFieldsReader, TestLucene50StoredFieldsFormat, TestFilterIterator, TestStressDeletes, TestIndexWriterOnDiskFull, TestForceMergeForever, MultiCollectorTest, TestNumericDocValuesUpdates, TestConsistentFieldNumbers, TestSimpleExplanationsOfNonMatches, TestPayloadExplanations, TestSegmentTermDocs, TestIsCurrent, TestNIOFSDirectory, TestMockAnalyzer, TestNumericRangeQuery32, TestMultiDocValues, TestLevenshteinAutomata, TestByteArrayDataInput, TestSortRandom, TestByteSlices, TestMultiMMap, Test2BTerms, TestIndexInput, TestDocValuesIndexing, TestDemoParallelLeafReader, TestAddIndexes, TestBoolean2, TestDemo, TestIndexWriterExceptions2, TestManyFields, TestDocumentsWriterStallControl, TestRegExp, TestFieldValueFilter, TestLogMergePolicy, TestSearcherManager, Nested1, Nested2]
   [junit4]   1> NOTE: reproduce with: ant test  -Dtestcase=Nested2 -Dtests.seed=C5F2CA80130DA348 -Dtests.slow=true -Dtests.locale=sr_CS -Dtests.timezone=Etc/UCT -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
   [junit4]   1> 
   [junit4] ERROR   0.04s | TestLeaveFilesIfTestFails.testWindowsUnremovableFile <<<
   [junit4]    > Throwable #1: java.nio.file.AccessDeniedException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\core\test\J0\temp\lucene.util.junitcompat.TestLeaveFilesIfTestFails$Nested2-C5F2CA80130DA348-001\leftover-001\child.locked
   [junit4]    > 	at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
   [junit4]    > 	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
   [junit4]    > 	at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
   [junit4]    > 	at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
   [junit4]    > 	at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
   [junit4]    > 	at org.apache.lucene.mockfile.WindowsFS.getKey(WindowsFS.java:57)
   [junit4]    > 	at org.apache.lucene.mockfile.WindowsFS.onClose(WindowsFS.java:77)
   [junit4]    > 	at org.apache.lucene.mockfile.HandleTrackingFS$5.close(HandleTrackingFS.java:210)
   [junit4]    > 	at org.apache.lucene.util.junitcompat.TestLeaveFilesIfTestFails.testWindowsUnremovableFile(TestLeaveFilesIfTestFails.java:79)
   [junit4] Completed in 0.10s, 2 tests, 1 error <<< FAILURES!

[...truncated 1156 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:515: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:463: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:61: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build.xml:49: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1349: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:956: There were test failures: 405 suites, 3053 tests, 1 error, 63 ignored (53 assumptions)

Total time: 10 minutes 43 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.8.0_40-ea-b09 -XX:+UseCompressedOops -XX:+UseG1GC (asserts: true)
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.8.0_40-ea-b09) - Build # 4463 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4463/
Java: 64bit/jdk1.8.0_40-ea-b09 -XX:+UseCompressedOops -XX:+UseParallelGC (asserts: false)

2 tests failed.
FAILED:  org.apache.solr.client.solrj.embedded.JettyWebappTest.testAdminUI

Error Message:
Server returned HTTP response code: 503 for URL: http://127.0.0.1:55057/test/

Stack Trace:
java.io.IOException: Server returned HTTP response code: 503 for URL: http://127.0.0.1:55057/test/
	at __randomizedtesting.SeedInfo.seed([614C5E38F26C5E76:599EBD6B9507C5C9]:0)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1838)
	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1439)
	at java.net.URL.openStream(URL.java:1038)
	at org.apache.solr.client.solrj.embedded.JettyWebappTest.testAdminUI(JettyWebappTest.java:102)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  junit.framework.TestSuite.org.apache.solr.client.solrj.embedded.JettyWebappTest

Error Message:
7 threads leaked from SUITE scope at org.apache.solr.client.solrj.embedded.JettyWebappTest:     1) Thread[id=113, name=qtp1117973912-113, state=TIMED_WAITING, group=TGRP-JettyWebappTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=114, name=qtp1117973912-114, state=TIMED_WAITING, group=TGRP-JettyWebappTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=112, name=qtp1117973912-112, state=TIMED_WAITING, group=TGRP-JettyWebappTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=116, name=qtp1117973912-116, state=TIMED_WAITING, group=TGRP-JettyWebappTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=111, name=qtp1117973912-111, state=TIMED_WAITING, group=TGRP-JettyWebappTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=115, name=qtp1117973912-115, state=TIMED_WAITING, group=TGRP-JettyWebappTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=117, name=qtp1117973912-117, state=TIMED_WAITING, group=TGRP-JettyWebappTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 7 threads leaked from SUITE scope at org.apache.solr.client.solrj.embedded.JettyWebappTest: 
   1) Thread[id=113, name=qtp1117973912-113, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=114, name=qtp1117973912-114, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=112, name=qtp1117973912-112, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=116, name=qtp1117973912-116, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=111, name=qtp1117973912-111, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=115, name=qtp1117973912-115, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=117, name=qtp1117973912-117, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([614C5E38F26C5E76]:0)




Build Log:
[...truncated 10478 lines...]
   [junit4] Suite: org.apache.solr.client.solrj.embedded.JettyWebappTest
   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.JettyWebappTest-614C5E38F26C5E76-001\init-core-data-001
   [junit4]   2> 33958 T109 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (true)
   [junit4]   2> 33960 T109 oas.SolrTestCaseJ4.setUp ###Starting testAdminUI
   [junit4]   2> 34383 T109 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 34613 T109 oejw.StandardDescriptorProcessor.visitServlet NO JSP Support for /test, did not find org.apache.jasper.servlet.JspServlet
   [junit4]   2> 34650 T109 oejw.WebAppContext.doStart WARN Failed startup of context o.e.j.w.WebAppContext{/test,file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/webapp/web/},C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\webapp\web java.lang.reflect.InvocationTargetException
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   [junit4]   2> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2> 	at org.eclipse.jetty.webapp.IterativeDescriptorProcessor.visit(IterativeDescriptorProcessor.java:85)
   [junit4]   2> 	at org.eclipse.jetty.webapp.IterativeDescriptorProcessor.process(IterativeDescriptorProcessor.java:72)
   [junit4]   2> 	at org.eclipse.jetty.webapp.MetaData.resolve(MetaData.java:367)
   [junit4]   2> 	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1250)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:710)
   [junit4]   2> 	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:494)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.doStart(HandlerWrapper.java:95)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.doStart(Server.java:280)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:64)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettyWebappTest.setUp(JettyWebappTest.java:81)
   [junit4]   2> 	at sun.reflect.GeneratedMethodAccessor12.invoke(Unknown Source)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:861)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> Caused by: java.lang.IllegalStateException: No such servlet: SolrConfigRestApi
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1274)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.setServletMappings(ServletHandler.java:1368)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.addServletMapping(ServletHandler.java:851)
   [junit4]   2> 	at org.eclipse.jetty.webapp.StandardDescriptorProcessor.addServletMapping(StandardDescriptorProcessor.java:1198)
   [junit4]   2> 	at org.eclipse.jetty.webapp.StandardDescriptorProcessor.visitServletMapping(StandardDescriptorProcessor.java:632)
   [junit4]   2> 	... 53 more
   [junit4]   2> 
   [junit4]   2> 34660 T109 oejs.AbstractConnector.doStart Started SocketConnector@0.0.0.0:55057
   [junit4]   2> 34763 T109 oejsh.ContextHandler.doStop stopped o.e.j.w.WebAppContext{/test,file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/webapp/web/},C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\webapp\web
   [junit4]   2> 34766 T109 oejuc.AbstractLifeCycle.setFailed WARN FAILED o.e.j.w.WebAppContext{/test,file:/C:/Users/JenkinsSlave/workspace/Lucene-Solr-trunk-Windows/solr/webapp/web/},C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\webapp\web: java.lang.IllegalStateException: No such servlet: SolrConfigRestApi java.lang.IllegalStateException: No such servlet: SolrConfigRestApi
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1274)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1345)
   [junit4]   2> 	at org.eclipse.jetty.webapp.WebXmlConfiguration.deconfigure(WebXmlConfiguration.java:129)
   [junit4]   2> 	at org.eclipse.jetty.webapp.WebAppContext.doStop(WebAppContext.java:523)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.doStop(HandlerWrapper.java:107)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.doStop(Server.java:341)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettyWebappTest.tearDown(JettyWebappTest.java:89)
   [junit4]   2> 	at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:885)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 34767 T109 oejuc.AbstractLifeCycle.setFailed WARN FAILED org.eclipse.jetty.server.Server@77de7979: java.lang.IllegalStateException: No such servlet: SolrConfigRestApi java.lang.IllegalStateException: No such servlet: SolrConfigRestApi
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1274)
   [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1345)
   [junit4]   2> 	at org.eclipse.jetty.webapp.WebXmlConfiguration.deconfigure(WebXmlConfiguration.java:129)
   [junit4]   2> 	at org.eclipse.jetty.webapp.WebAppContext.doStop(WebAppContext.java:523)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2> 	at org.eclipse.jetty.server.handler.HandlerWrapper.doStop(HandlerWrapper.java:107)
   [junit4]   2> 	at org.eclipse.jetty.server.Server.doStop(Server.java:341)
   [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.stop(AbstractLifeCycle.java:89)
   [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettyWebappTest.tearDown(JettyWebappTest.java:89)
   [junit4]   2> 	at sun.reflect.GeneratedMethodAccessor13.invoke(Unknown Source)
   [junit4]   2> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
   [junit4]   2> 	at java.lang.reflect.Method.invoke(Method.java:497)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:885)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
   [junit4]   2> 	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
   [junit4]   2> 	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
   [junit4]   2> 	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 34767 T109 oas.SolrTestCaseJ4.tearDown ###Ending testAdminUI
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=JettyWebappTest -Dtests.method=testAdminUI -Dtests.seed=614C5E38F26C5E76 -Dtests.slow=true -Dtests.locale=sv_SE -Dtests.timezone=America/Indianapolis -Dtests.asserts=false -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.89s | JettyWebappTest.testAdminUI <<<
   [junit4]    > Throwable #1: java.io.IOException: Server returned HTTP response code: 503 for URL: http://127.0.0.1:55057/test/
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([614C5E38F26C5E76:599EBD6B9507C5C9]:0)
   [junit4]    > 	at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1838)
   [junit4]    > 	at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1439)
   [junit4]    > 	at java.net.URL.openStream(URL.java:1038)
   [junit4]    > 	at org.apache.solr.client.solrj.embedded.JettyWebappTest.testAdminUI(JettyWebappTest.java:102)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 34845 T109 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-solrj\test\J0\temp\solr.client.solrj.embedded.JettyWebappTest-614C5E38F26C5E76-001
   [junit4]   2> 892 T108 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 7 leaked thread(s).
   [junit4]   2> 21025 T108 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 7 threads leaked from SUITE scope at org.apache.solr.client.solrj.embedded.JettyWebappTest: 
   [junit4]   2> 	   1) Thread[id=113, name=qtp1117973912-113, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2> 	        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   2) Thread[id=114, name=qtp1117973912-114, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2> 	        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   3) Thread[id=112, name=qtp1117973912-112, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2> 	        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   4) Thread[id=116, name=qtp1117973912-116, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2> 	        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   5) Thread[id=111, name=qtp1117973912-111, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2> 	        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   6) Thread[id=115, name=qtp1117973912-115, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2> 	        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 	   7) Thread[id=117, name=qtp1117973912-117, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	        at sun.misc.Unsafe.park(Native Method)
   [junit4]   2> 	        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]   2> 	        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]   2> 	        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]   2> 	        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]   2> 	        at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 21026 T108 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
   [junit4]   2> 	   1) Thread[id=113, name=qtp1117973912-113, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	   2) Thread[id=114, name=qtp1117973912-114, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	   3) Thread[id=112, name=qtp1117973912-112, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	   4) Thread[id=116, name=qtp1117973912-116, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	   5) Thread[id=111, name=qtp1117973912-111, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	   6) Thread[id=115, name=qtp1117973912-115, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 	   7) Thread[id=117, name=qtp1117973912-117, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]   2> 21028 T108 ccr.ThreadLeakControl.tryToInterruptAll All leaked threads terminated.
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=sv_SE, timezone=America/Indianapolis
   [junit4]   2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.8.0_40-ea (64-bit)/cpus=2,threads=1,free=100345544,total=207093760
   [junit4]   2> NOTE: All tests run in this JVM: [ExternalHttpClientTest, ContentStreamTest, TestDocumentObjectBinder, LargeVolumeEmbeddedTest, SolrExampleBinaryTest, TestNamedListCodec, TestRetryUtil, TermsResponseTest, ModifiableSolrParamsTest, MergeIndexesEmbeddedTest, MultiCoreEmbeddedTest, TestHash, DocumentAnalysisResponseTest, SolrDocumentTest, JettyWebappTest]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=JettyWebappTest -Dtests.seed=614C5E38F26C5E76 -Dtests.slow=true -Dtests.locale=sv_SE -Dtests.timezone=America/Indianapolis -Dtests.asserts=false -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s | JettyWebappTest (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 7 threads leaked from SUITE scope at org.apache.solr.client.solrj.embedded.JettyWebappTest: 
   [junit4]    >    1) Thread[id=113, name=qtp1117973912-113, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    2) Thread[id=114, name=qtp1117973912-114, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    3) Thread[id=112, name=qtp1117973912-112, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    4) Thread[id=116, name=qtp1117973912-116, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    5) Thread[id=111, name=qtp1117973912-111, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    6) Thread[id=115, name=qtp1117973912-115, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    7) Thread[id=117, name=qtp1117973912-117, state=TIMED_WAITING, group=TGRP-JettyWebappTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
   [junit4]    >         at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:342)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:526)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool.access$600(QueuedThreadPool.java:44)
   [junit4]    >         at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([614C5E38F26C5E76]:0)
   [junit4] Completed in 21.06s, 1 test, 2 errors <<< FAILURES!

[...truncated 128 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:519: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:467: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:61: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:202: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:510: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1349: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:956: There were test failures: 54 suites, 330 tests, 1 suite-level error, 1 error

Total time: 145 minutes 33 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.8.0_40-ea-b09 -XX:+UseCompressedOops -XX:+UseParallelGC (asserts: false)
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.8.0_20) - Build # 4462 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4462/
Java: 64bit/jdk1.8.0_20 -XX:-UseCompressedOops -XX:+UseG1GC (asserts: false)

3 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.LeaderInitiatedRecoveryOnCommitTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.LeaderInitiatedRecoveryOnCommitTest:     1) Thread[id=2541, name=SessionTracker, state=TIMED_WAITING, group=TGRP-LeaderInitiatedRecoveryOnCommitTest]         at java.lang.Object.wait(Native Method)         at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.LeaderInitiatedRecoveryOnCommitTest: 
   1) Thread[id=2541, name=SessionTracker, state=TIMED_WAITING, group=TGRP-LeaderInitiatedRecoveryOnCommitTest]
        at java.lang.Object.wait(Native Method)
        at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
	at __randomizedtesting.SeedInfo.seed([A04E2CBCA5501A09]:0)


FAILED:  org.apache.solr.handler.TestReplicationHandlerBackup.doTestBackup

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([A04E2CBCA5501A09]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([A04E2CBCA5501A09]:0)




Build Log:
[...truncated 8898 lines...]
   [junit4] Suite: org.apache.solr.handler.TestReplicationHandlerBackup
   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\init-core-data-001
   [junit4]   2> 518677 T1322 oas.SolrTestCaseJ4.setUp ###Starting doTestBackup
   [junit4]   2> 518688 T1322 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 518713 T1322 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:58429
   [junit4]   2> 518714 T1322 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 518714 T1322 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 518715 T1322 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001
   [junit4]   2> 518715 T1322 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\'
   [junit4]   2> 518744 T1322 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\solr.xml
   [junit4]   2> 518755 T1322 oasc.CoreContainer.<init> New CoreContainer 1797947065
   [junit4]   2> 518755 T1322 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\]
   [junit4]   2> 518755 T1322 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 518756 T1322 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 518756 T1322 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 518756 T1322 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 518757 T1322 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 518757 T1322 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 518757 T1322 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 518757 T1322 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 518757 T1322 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 518757 T1322 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 518758 T1322 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 518758 T1322 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 518760 T1322 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 518760 T1322 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 518766 T1332 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\'
   [junit4]   2> 518788 T1332 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 518808 T1332 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 518810 T1332 oass.IndexSchema.readSchema Reading Solr Schema from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\conf\schema.xml
   [junit4]   2> 518811 T1332 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 518813 T1332 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 518813 T1332 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from instancedir C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\
   [junit4]   2> 518813 T1332 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 518814 T1332 oasc.SolrCore.<init> [collection1] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\data\
   [junit4]   2> 518814 T1332 oasc.SolrCore.<init> JMX monitoring not detected for core: collection1
   [junit4]   2> 518814 T1332 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\data\
   [junit4]   2> 518814 T1332 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\data\index/
   [junit4]   2> 518814 T1332 oasc.SolrCore.initIndex WARN [collection1] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\data\index' doesn't exist. Creating new index...
   [junit4]   2> 518814 T1332 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\data\index
   [junit4]   2> 518814 T1332 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=23, maxMergeAtOnceExplicit=15, maxMergedSegmentMB=81.30859375, floorSegmentMB=1.8857421875, forceMergeDeletesPctAllowed=2.4175444507422474, segmentsPerTier=28.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0
   [junit4]   2> 518815 T1332 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@68a2f7b2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6dc123f7),segFN=segments_1,generation=1}
   [junit4]   2> 518815 T1332 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 518816 T1332 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 518817 T1332 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 518817 T1332 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 518817 T1332 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 518817 T1332 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
   [junit4]   2> 518817 T1332 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler
   [junit4]   2> 518817 T1332 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
   [junit4]   2> 518818 T1332 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
   [junit4]   2> 518818 T1332 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
   [junit4]   2> 518818 T1332 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
   [junit4]   2> 518819 T1332 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
   [junit4]   2> 518819 T1332 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler
   [junit4]   2> 518819 T1332 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
   [junit4]   2> 518819 T1332 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: org.apache.solr.handler.admin.ShowFileRequestHandler
   [junit4]   2> 518819 T1332 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler
   [junit4]   2> 518819 T1332 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 518832 T1332 oasc.RequestHandlers.initHandlersFromConfig created /replication: solr.ReplicationHandler
   [junit4]   2> 518832 T1332 oasc.RequestHandlers.initHandlersFromConfig created defaults: solr.StandardRequestHandler
   [junit4]   2> 518832 T1332 oasc.RequestHandlers.initHandlersFromConfig adding lazy requestHandler: solr.StandardRequestHandler
   [junit4]   2> 518832 T1332 oasc.RequestHandlers.initHandlersFromConfig created lazy: solr.StandardRequestHandler
   [junit4]   2> 518833 T1332 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 518837 T1332 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 518839 T1332 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 518839 T1332 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 518841 T1332 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 518841 T1332 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 518841 T1332 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 518842 T1332 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=42, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=false, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0]
   [junit4]   2> 518842 T1332 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@68a2f7b2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6dc123f7),segFN=segments_1,generation=1}
   [junit4]   2> 518843 T1332 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 518843 T1332 oass.SolrIndexSearcher.<init> Opening Searcher@6ad4d6d3[collection1] main
   [junit4]   2> 518843 T1332 oasr.ManagedResourceStorage$FileStorageIO.configure File-based storage initialized to use dir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\conf
   [junit4]   2> 518843 T1332 oasr.RestManager.init Initializing RestManager with initArgs: {storageDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\conf}
   [junit4]   2> 518843 T1332 oasr.ManagedResourceStorage.load Reading _rest_managed.json using file:dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\conf
   [junit4]   2> 518844 T1332 oasr.ManagedResource.reloadFromStorage WARN No stored data found for /rest/managed
   [junit4]   2> 518845 T1332 oasr.ManagedResourceStorage$JsonStorage.store Saved JSON object to path _rest_managed.json using file:dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.handler.TestReplicationHandlerBackup-A04E2CBCA5501A09-001\solr-instance-001\collection1\conf
   [junit4]   2> 518845 T1332 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 518845 T1332 oash.ReplicationHandler.inform Replication enabled for following config files: schema-replication2.xml:schema.xml
   [junit4]   2> 518845 T1332 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 518845 T1332 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 518845 T1333 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@6ad4d6d3[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 518846 T1322 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 518846 T1322 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> ASYNC  NEW_CORE C200 name=collection1 org.apache.solr.core.SolrCore@4e47c1fe
   [junit4]   2> 518851 T1326 C200 oasu.DirectUpdateHandler2.deleteAll [collection1] REMOVING ALL DOCUMENTS FROM INDEX
   [junit4]   2> 518853 T1326 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {deleteByQuery=*:*} 0 4
   [junit4]   2> 518857 T1327 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[0]} 0 1
   [junit4]   2> 518860 T1328 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[1]} 0 0
   [junit4]   2> 518862 T1329 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[2]} 0 0
   [junit4]   2> 518864 T1330 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[3]} 0 0
   [junit4]   2> 518866 T1331 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[4]} 0 0
   [junit4]   2> 518868 T1326 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[5]} 0 0
   [junit4]   2> 518871 T1327 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[6]} 0 1
   [junit4]   2> 518873 T1328 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[7]} 0 0
   [junit4]   2> 518875 T1329 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[8]} 0 0
   [junit4]   2> 518877 T1330 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[9]} 0 0
   [junit4]   2> 518879 T1331 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[10]} 0 0
   [junit4]   2> 518881 T1326 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[11]} 0 0
   [junit4]   2> 518883 T1327 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[12]} 0 0
   [junit4]   2> 518885 T1328 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[13]} 0 0
   [junit4]   2> 518887 T1329 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[14]} 0 0
   [junit4]   2> 518889 T1330 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[15]} 0 0
   [junit4]   2> 518891 T1331 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[16]} 0 0
   [junit4]   2> 518893 T1326 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[17]} 0 0
   [junit4]   2> 518896 T1327 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[18]} 0 0
   [junit4]   2> 518898 T1328 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[19]} 0 0
   [junit4]   2> 518900 T1329 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[20]} 0 0
   [junit4]   2> 518902 T1330 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[21]} 0 0
   [junit4]   2> 518904 T1331 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[22]} 0 0
   [junit4]   2> 518907 T1326 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[23]} 0 0
   [junit4]   2> 518909 T1327 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[24]} 0 0
   [junit4]   2> 518911 T1328 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[25]} 0 0
   [junit4]   2> 518913 T1329 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[26]} 0 0
   [junit4]   2> 518915 T1330 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[27]} 0 0
   [junit4]   2> 518917 T1331 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[28]} 0 0
   [junit4]   2> 518919 T1326 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[29]} 0 0
   [junit4]   2> 518921 T1327 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[30]} 0 0
   [junit4]   2> 518923 T1328 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[31]} 0 0
   [junit4]   2> 518925 T1329 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[32]} 0 0
   [junit4]   2> 518926 T1330 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[33]} 0 0
   [junit4]   2> 518928 T1331 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[34]} 0 0
   [junit4]   2> 518931 T1326 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[35]} 0 0
   [junit4]   2> 518932 T1327 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[36]} 0 0
   [junit4]   2> 518933 T1328 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[37]} 0 0
   [junit4]   2> 518935 T1329 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[38]} 0 0
   [junit4]   2> 518936 T1330 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[39]} 0 0
   [junit4]   2> 518938 T1331 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[40]} 0 0
   [junit4]   2> 518939 T1326 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[41]} 0 0
   [junit4]   2> 518942 T1327 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[42]} 0 0
   [junit4]   2> 518944 T1328 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[43]} 0 0
   [junit4]   2> 518946 T1329 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[44]} 0 0
   [junit4]   2> 518948 T1330 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[45]} 0 0
   [junit4]   2> 518950 T1331 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[46]} 0 0
   [junit4]   2> 518952 T1326 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[47]} 0 0
   [junit4]   2> 518954 T1327 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[48]} 0 0
   [junit4]   2> 518956 T1328 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[49]} 0 0
   [junit4]   2> 518958 T1329 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[50]} 0 0
   [junit4]   2> 518960 T1330 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[51]} 0 0
   [junit4]   2> 518963 T1331 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[52]} 0 0
   [junit4]   2> 518965 T1326 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[53]} 0 0
   [junit4]   2> 518967 T1327 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[54]} 0 0
   [junit4]   2> 518970 T1328 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[55]} 0 0
   [junit4]   2> 518972 T1329 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={wt=javabin&version=2} {add=[56]} 0 0
   [junit4]   2> 518974 T1330 C200 oasu.DirectUpdateHandler2.commit start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=false,prepareCommit=false}
   [junit4]   2> 518977 T1330 C200 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=2
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@68a2f7b2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6dc123f7),segFN=segments_1,generation=1}
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@68a2f7b2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@6dc123f7),segFN=segments_2,generation=2}
   [junit4]   2> 518978 T1330 C200 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 2
   [junit4]   2> 518979 T1330 C200 oass.SolrIndexSearcher.<init> Opening Searcher@2335b24[collection1] main
   [junit4]   2> 518979 T1330 C200 oasu.DirectUpdateHandler2.commit end_commit_flush
   [junit4]   2> 518979 T1333 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@2335b24[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader(Uninverting(_0(6.0.0):C57)))}
   [junit4]   2> 518979 T1330 C200 oasup.LogUpdateProcessor.finish [collection1] webapp=/solr path=/update params={waitSearcher=true&commit=true&softCommit=false&wt=javabin&version=2} {commit=} 0 5
   [junit4]   2> 519040 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={name=aeavo&command=backup} status=0 QTime=9 
   [junit4]   2> 519047 T1335 oash.SnapShooter.createSnapshot Creating backup snapshot...
   [junit4]   2> 519057 T1335 oash.SnapShooter.createSnapshot Done creating backup snapshot: aeavo
   [junit4]   2> 519058 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=5 
   [junit4]   2> 519200 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 519914 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={name=gapqmz&command=backup} status=0 QTime=1 
   [junit4]   2> 519915 T1337 oash.SnapShooter.createSnapshot Creating backup snapshot...
   [junit4]   2> 519922 T1337 oash.SnapShooter.createSnapshot Done creating backup snapshot: gapqmz
   [junit4]   2> 519922 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=5 
   [junit4]   2> 520065 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 520074 T1338 oash.SnapShooter.deleteNamedSnapshot Deleting snapshot: aeavo
   [junit4]   2> 520075 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={name=aeavo&command=deletebackup} status=0 QTime=4 
   [junit4]   2> 520079 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 520790 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={name=gapqmz&command=deletebackup} status=0 QTime=1 
   [junit4]   2> 520790 T1339 oash.SnapShooter.deleteNamedSnapshot Deleting snapshot: gapqmz
   [junit4]   2> 520794 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 520937 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 521079 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 521222 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 521364 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 521506 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 521658 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 521816 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 521974 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 522131 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 522288 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 522444 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 522601 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 522758 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 522915 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 523072 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 523230 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 523388 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 523545 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 523702 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 523858 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 524015 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 524173 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 524330 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 524488 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 524645 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 524807 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 524964 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 525121 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 525278 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 525436 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 525593 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 525751 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 525908 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 526066 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 526223 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 526380 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 526537 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 526693 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 526851 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 527008 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 527164 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 527320 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 527477 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 527634 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 527792 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 527950 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 528105 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 528262 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 528418 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 528574 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 528731 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 528888 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 529044 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 529201 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 529373 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 529545 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 529717 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 529889 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 530061 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 530234 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 530406 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 530578 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 530750 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 530922 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 531093 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 531266 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 531438 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 531610 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 531782 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 531954 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 532126 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 532298 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 532470 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 532642 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 532814 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 532986 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 533158 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 533330 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 533502 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 533675 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 533847 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 534019 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 534191 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 534363 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 534535 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 534706 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 534879 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 535052 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 535224 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 535395 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 535568 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 535740 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 535912 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 536084 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 536256 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 536428 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 536600 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 536772 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 536944 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 537116 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 537288 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 537460 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 537632 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 537804 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 537992 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 538182 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 538371 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 538559 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 538749 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 538940 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 539130 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 539320 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 539508 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 539697 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 539886 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 540076 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 540265 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 540454 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 540643 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 540833 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 541021 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 541210 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 541400 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 541589 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 541778 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 541967 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 542156 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 542345 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 542537 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 542726 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 542915 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 543105 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 543294 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 543482 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 543670 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 543860 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 544050 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 544239 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 544429 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 544618 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 544807 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 544996 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 545186 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 545375 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 545564 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 545753 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 545942 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 546132 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 546321 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 546510 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 546699 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 546888 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 547078 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 547277 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 547485 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 547692 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 547900 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 548107 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 548314 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 548522 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 548731 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 548941 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 549151 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 549359 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 549566 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 549774 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 549983 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 550191 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 550399 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 550608 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 550816 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 551024 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 551232 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 551440 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 551648 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 551856 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 552064 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 552272 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 552481 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 552689 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 552897 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 553105 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 553313 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 553520 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 553729 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 553936 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 554146 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 554353 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 554562 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 554770 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 554978 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 555186 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 555394 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 555601 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 555810 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 556020 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 556228 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 556435 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 556644 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 556852 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 557060 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 557268 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 557478 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 557708 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 557937 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 558165 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 558396 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 558624 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 558853 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 559082 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 559311 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 559545 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 559777 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 560006 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 560235 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 560465 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 560694 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 560924 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 561154 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 561383 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 561613 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 561842 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 562072 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 562300 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 562529 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 562758 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 562987 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 563215 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 563445 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 563674 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 563902 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 564132 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 564360 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 564590 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 564819 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 565048 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 565278 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 565508 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 565738 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 565967 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 566197 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 566437 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 566665 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 566895 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 567126 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 567358 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 567590 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 567828 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 568058 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 568289 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 568524 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 568754 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 569004 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 569258 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 569510 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 569762 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 570014 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 570266 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 570517 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 570769 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 571021 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 571273 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 571526 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 571778 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 572030 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 572283 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 572536 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 572787 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 573039 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 573291 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 573543 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 573795 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 574046 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 574298 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 574549 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 574801 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 575051 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 575303 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 575556 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 575808 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 576060 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 576312 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 576583 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 576835 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 577087 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 577338 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 577590 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 577841 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 578093 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 578344 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 578596 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 578849 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 579101 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 579352 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 579605 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 579856 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 580108 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 580360 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 580612 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 580864 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 581116 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 581378 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 581657 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 581934 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 582211 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 582487 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 582764 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 583040 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 583317 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 583594 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 583872 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 584148 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 584425 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 584703 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 584980 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 585256 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 585533 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 585810 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 586087 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 586364 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 586641 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 586917 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 587194 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 587471 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 587749 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 588026 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 588302 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 588577 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 588854 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 589130 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 589407 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 589681 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 589958 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 590234 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 590511 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 590788 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 591065 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 591340 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 591617 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 591894 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 592171 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 592448 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 592725 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 593001 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 593277 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 593554 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 593829 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 594106 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 594382 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 594658 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 594934 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 595235 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 595540 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 595843 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 596146 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 596449 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 596753 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 597056 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 597359 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 597662 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 597965 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 598270 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 598573 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 598876 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 599181 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 599484 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 599788 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 600091 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 600394 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 600697 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 601000 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 601304 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 601608 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 601912 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 602215 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 602518 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 602821 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 603123 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 603426 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 603729 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 604033 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 604336 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 604639 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 604942 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 605245 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 605547 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 605850 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 606152 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 606455 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 606757 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 607060 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 607364 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 607667 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 607970 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 608272 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 608575 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 608878 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 609181 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 609485 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 609786 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 610090 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 610391 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 610693 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 610998 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 611301 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 611604 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 611907 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 612212 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 612515 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 612818 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 613122 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 613425 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 613728 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 614030 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 614333 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 614635 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 614938 T1331 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 615241 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 615544 T1329 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 615846 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 616149 T1327 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=1 
   [junit4]   2> 616451 T1328 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=0 
   [junit4]   2> 616756 T1330 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status=0 QTime=2 
   [junit4]   2> 617059 T1326 C200 oasc.SolrCore.execute [collection1] webapp=/solr path=/replication params={command=details} status

[...truncated too long message...]

eakControl$StatementRunner.run(ThreadLeakControl.java:365)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 7657696 T2538 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 7657696 T2538 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.LeaderInitiatedRecoveryOnCommitTest-A04E2CBCA5501A09-001\tempDir-001/jetty4\index [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.LeaderInitiatedRecoveryOnCommitTest-A04E2CBCA5501A09-001\tempDir-001/jetty4\index;done=false>>]
   [junit4]   2> 7657696 T2538 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.LeaderInitiatedRecoveryOnCommitTest-A04E2CBCA5501A09-001\tempDir-001/jetty4\index
   [junit4]   2> 7657696 T2538 oasc.CachingDirectoryFactory.closeCacheValue looking to close C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.LeaderInitiatedRecoveryOnCommitTest-A04E2CBCA5501A09-001\tempDir-001/jetty4\ [CachedDir<<refCount=0;path=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.LeaderInitiatedRecoveryOnCommitTest-A04E2CBCA5501A09-001\tempDir-001/jetty4\;done=false>>]
   [junit4]   2> 7657697 T2538 oasc.CachingDirectoryFactory.close Closing directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.LeaderInitiatedRecoveryOnCommitTest-A04E2CBCA5501A09-001\tempDir-001/jetty4\
   [junit4]   2> 7657698 T2538 oasc.Overseer.close Overseer (id=92890117425856524-127.0.0.1:59056_-n_0000000004) closing
   [junit4]   2> 7657701 T2834 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:59056_
   [junit4]   2> 7659849 T2832 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 7659875 T2538 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
   [junit4]   2> 7667182 T2538 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 59005
   [junit4]   2> 7667182 T2538 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
   [junit4]   2> 7667185 T2538 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:58998 58998
   [junit4]   2> 7670486 T2539 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:58998 58998
   [junit4]   2> 7670493 T2539 oasc.ZkTestServer$ZKServerMain.runFromConfig WARN Watch limit violations: 
   [junit4]   2> 	Maximum concurrent create/delete watches above limit:
   [junit4]   2> 	
   [junit4]   2> 		6	/solr/aliases.json
   [junit4]   2> 		6	/solr/clusterstate.json
   [junit4]   2> 		5	/solr/configs/conf1
   [junit4]   2> 	
   [junit4]   2> 	Maximum concurrent children watches above limit:
   [junit4]   2> 	
   [junit4]   2> 		6	/solr/live_nodes
   [junit4]   2> 		5	/solr/overseer/collection-queue-work
   [junit4]   2> 		4	/solr/overseer/queue
   [junit4]   2> 	
   [junit4]   2> 7670494 T2538 oasc.SocketProxy.close WARN Closing 0 connections to: http://127.0.0.1:59040/
   [junit4]   2> 7670494 T2538 oasc.SocketProxy.close WARN Closing 0 connections to: http://127.0.0.1:59056/
   [junit4]   2> 7670496 T2538 oasc.SocketProxy.close WARN Closing 0 connections to: http://127.0.0.1:59005/
   [junit4]   2> 7670497 T2538 oasc.SocketProxy.close WARN Closing 0 connections to: http://127.0.0.1:59028/
   [junit4]   2> 7670497 T2538 oasc.SocketProxy.close WARN Closing 0 connections to: http://127.0.0.1:59018/
   [junit4]   2> 7670499 T2538 oasc.SocketProxy.close WARN Closing 0 connections to: http://127.0.0.1:59040/
   [junit4]   2> 7670500 T2538 oasc.SocketProxy.close WARN Closing 0 connections to: http://127.0.0.1:59056/
   [junit4]   2> 7670500 T2538 oasc.SocketProxy.close WARN Closing 0 connections to: http://127.0.0.1:59005/
   [junit4]   2> 7670500 T2538 oasc.SocketProxy.close WARN Closing 0 connections to: http://127.0.0.1:59028/
   [junit4]   2> 7670500 T2538 oasc.SocketProxy.close WARN Closing 0 connections to: http://127.0.0.1:59018/
   [junit4]   2> 7670502 T2538 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> -508974 T2537 ccr.ThreadLeakControl.checkThreadLeaks WARNING Will linger awaiting termination of 1 leaked thread(s).
   [junit4]   2> -488610 T2537 ccr.ThreadLeakControl.checkThreadLeaks SEVERE 1 thread leaked from SUITE scope at org.apache.solr.cloud.LeaderInitiatedRecoveryOnCommitTest: 
   [junit4]   2> 	   1) Thread[id=2541, name=SessionTracker, state=TIMED_WAITING, group=TGRP-LeaderInitiatedRecoveryOnCommitTest]
   [junit4]   2> 	        at java.lang.Object.wait(Native Method)
   [junit4]   2> 	        at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
   [junit4]   2> -488610 T2537 ccr.ThreadLeakControl.tryToInterruptAll Starting to interrupt leaked threads:
   [junit4]   2> 	   1) Thread[id=2541, name=SessionTracker, state=TIMED_WAITING, group=TGRP-LeaderInitiatedRecoveryOnCommitTest]
   [junit4]   2> 7694040 T2541 oazs.SessionTrackerImpl.run ERROR Unexpected interruption java.lang.InterruptedException
   [junit4]   2> 	at java.lang.Object.wait(Native Method)
   [junit4]   2> 	at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
   [junit4]   2> 
   [junit4]   2> -488606 T2537 ccr.ThreadLeakControl.tryToInterruptAll All leaked threads terminated.
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50), sim=DefaultSimilarity, locale=uk, timezone=Pacific/Easter
   [junit4]   2> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.8.0_20 (64-bit)/cpus=2,threads=1,free=88045024,total=381681664
   [junit4]   2> NOTE: All tests run in this JVM: [BlockDirectoryTest, SchemaVersionSpecificBehaviorTest, ResourceLoaderTest, TestCharFilters, CoreMergeIndexesAdminHandlerTest, TestCollationField, LukeRequestHandlerTest, TestMissingGroups, OutputWriterTest, FileUtilsTest, TestPivotHelperCode, MultiTermTest, RAMDirectoryFactoryTest, SuggestComponentTest, TestLuceneMatchVersion, SyncSliceTest, TestAnalyzeInfixSuggestions, DistanceFunctionTest, TestNonNRTOpen, TestStressUserVersions, PreAnalyzedUpdateProcessorTest, TestGroupingSearch, ChangedSchemaMergeTest, TestPartialUpdateDeduplication, TestCoreContainer, ChaosMonkeyNothingIsSafeTest, SpellingQueryConverterTest, HttpPartitionTest, BadCopyFieldTest, AddBlockUpdateTest, TestCollectionAPI, TestDynamicFieldResource, BasicFunctionalityTest, TestSolrXml, BJQParserTest, ResponseHeaderTest, TestCustomSort, CursorPagingTest, ZkControllerTest, TestPseudoReturnFields, TestSolrDeletionPolicy1, ResponseLogComponentTest, DistributedExpandComponentTest, TestReplicationHandlerBackup, TestFastWriter, ExternalCollectionsTest, DistributedDebugComponentTest, MinimalSchemaTest, TestLFUCache, TestStressReorder, LeaderElectionTest, CurrencyFieldOpenExchangeTest, SpellCheckComponentTest, SolrCmdDistributorTest, TestUniqueKeyFieldResource, DebugComponentTest, TestSimpleQParserPlugin, OverseerRolesTest, PeerSyncTest, TestSchemaVersionResource, DateMathParserTest, MigrateRouteKeyTest, TestIBSimilarityFactory, ParsingFieldUpdateProcessorsTest, TestSolrXmlPersistence, TestConfigReload, TestSchemaResource, LeaderInitiatedRecoveryOnCommitTest]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=LeaderInitiatedRecoveryOnCommitTest -Dtests.seed=A04E2CBCA5501A09 -Dtests.slow=true -Dtests.locale=uk -Dtests.timezone=Pacific/Easter -Dtests.asserts=false -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s | LeaderInitiatedRecoveryOnCommitTest (suite) <<<
   [junit4]    > Throwable #1: com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.LeaderInitiatedRecoveryOnCommitTest: 
   [junit4]    >    1) Thread[id=2541, name=SessionTracker, state=TIMED_WAITING, group=TGRP-LeaderInitiatedRecoveryOnCommitTest]
   [junit4]    >         at java.lang.Object.wait(Native Method)
   [junit4]    >         at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([A04E2CBCA5501A09]:0)
   [junit4] Completed in -488.55s, 1 test, 1 error <<< FAILURES!

[...truncated 1237 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:519: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:467: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:61: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:189: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:510: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1349: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:956: There were test failures: 451 suites, 1840 tests, 2 suite-level errors, 1 error, 56 ignored (31 assumptions)

Total time: 270 minutes 45 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.8.0_20 -XX:-UseCompressedOops -XX:+UseG1GC (asserts: false)
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-trunk-Windows (32bit/jdk1.8.0_20) - Build # 4461 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4461/
Java: 32bit/jdk1.8.0_20 -server -XX:+UseParallelGC (asserts: false)

4 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.DeleteInactiveReplicaTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.DeleteInactiveReplicaTest:     1) Thread[id=1884, name=SessionTracker, state=TIMED_WAITING, group=TGRP-DeleteInactiveReplicaTest]         at java.lang.Object.wait(Native Method)         at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.DeleteInactiveReplicaTest: 
   1) Thread[id=1884, name=SessionTracker, state=TIMED_WAITING, group=TGRP-DeleteInactiveReplicaTest]
        at java.lang.Object.wait(Native Method)
        at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
	at __randomizedtesting.SeedInfo.seed([4F62D3D25588F392]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.cloud.DeleteReplicaTest

Error Message:
1 thread leaked from SUITE scope at org.apache.solr.cloud.DeleteReplicaTest:     1) Thread[id=13675, name=SessionTracker, state=TIMED_WAITING, group=TGRP-DeleteReplicaTest]         at java.lang.Object.wait(Native Method)         at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 1 thread leaked from SUITE scope at org.apache.solr.cloud.DeleteReplicaTest: 
   1) Thread[id=13675, name=SessionTracker, state=TIMED_WAITING, group=TGRP-DeleteReplicaTest]
        at java.lang.Object.wait(Native Method)
        at org.apache.zookeeper.server.SessionTrackerImpl.run(SessionTrackerImpl.java:146)
	at __randomizedtesting.SeedInfo.seed([4F62D3D25588F392]:0)


FAILED:  org.apache.solr.handler.TestReplicationHandlerBackup.doTestBackup

Error Message:
Test abandoned because suite timeout was reached.

Stack Trace:
java.lang.Exception: Test abandoned because suite timeout was reached.
	at __randomizedtesting.SeedInfo.seed([4F62D3D25588F392]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.handler.TestReplicationHandlerBackup

Error Message:
Suite timeout exceeded (>= 7200000 msec).

Stack Trace:
java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
	at __randomizedtesting.SeedInfo.seed([4F62D3D25588F392]:0)




Build Log:
[...truncated 8643 lines...]
   [junit4] Suite: org.apache.solr.cloud.DeleteInactiveReplicaTest
   [junit4]   2> Creating dataDir: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\init-core-data-001
   [junit4]   2> 662906 T1881 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (true) and clientAuth (false)
   [junit4]   2> 662906 T1881 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /p_gg/dv
   [junit4]   2> 662908 T1881 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 662908 T1881 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 662909 T1882 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 663010 T1881 oasc.ZkTestServer.run start zk server on port:50887
   [junit4]   2> 663010 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 663012 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 663016 T1889 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c4c909 name:ZooKeeperConnection Watcher:127.0.0.1:50887 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 663016 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 663016 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 663016 T1881 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 663023 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 663025 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 663028 T1892 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1b55e8b name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 663029 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 663029 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 663029 T1881 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 663032 T1881 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 663035 T1881 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 663037 T1881 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 663039 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 663039 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 663043 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 663043 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 663046 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 663046 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 663049 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 663049 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 663052 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 663052 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 663055 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 663055 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 663058 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 663058 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 663061 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 663061 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 663064 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 663064 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 663067 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 663067 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 663070 T1881 oasc.AbstractZkTestCase.putConfig put C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\core\src\test-files\solr\collection1\conf\synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 663070 T1881 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 663074 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 663076 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 663081 T1895 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1997f53 name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 663081 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 663081 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 664215 T1881 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 664223 T1881 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 664224 T1881 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:50897
   [junit4]   2> 664228 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 664228 T1881 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 664229 T1881 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-002
   [junit4]   2> 664229 T1881 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-002\'
   [junit4]   2> 664255 T1881 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-002\solr.xml
   [junit4]   2> 664266 T1881 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-002
   [junit4]   2> 664267 T1881 oasc.CoreContainer.<init> New CoreContainer 14486680
   [junit4]   2> 664267 T1881 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-002\]
   [junit4]   2> 664267 T1881 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 664268 T1881 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 664268 T1881 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 664269 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 664269 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 664269 T1881 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 664269 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 664270 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 664270 T1881 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 664270 T1881 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 664274 T1881 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 664277 T1881 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 664277 T1881 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 664277 T1881 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 664278 T1881 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50887/solr
   [junit4]   2> 664278 T1881 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 664278 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 664290 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 664294 T1907 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1a185aa name:ZooKeeperConnection Watcher:127.0.0.1:50887 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 664295 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 664295 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 664301 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 664304 T1910 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1855eb0 name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 664304 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 664307 T1881 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 664312 T1881 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 664317 T1881 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 664322 T1881 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 664326 T1881 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 664333 T1881 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 664336 T1881 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50897_p_gg%2Fdv
   [junit4]   2> 664337 T1881 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50897_p_gg%2Fdv
   [junit4]   2> 664341 T1881 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 664345 T1881 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 664348 T1881 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 664351 T1881 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:50897_p_gg%2Fdv
   [junit4]   2> 664352 T1881 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 664355 T1881 oasc.Overseer.start Overseer (id=92886067165528068-127.0.0.1:50897_p_gg%2Fdv-n_0000000000) starting
   [junit4]   2> 664359 T1881 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 664377 T1881 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=10000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 664380 T1881 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 664380 T1912 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 664384 T1881 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 664387 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 664393 T1911 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 664393 T1881 oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-002
   [junit4]   2> 664701 T1881 oasc.CorePropertiesLocator.discoverUnder Found core conf in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-002\conf\
   [junit4]   2> 664756 T1881 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 664758 T1881 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 664758 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 664778 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 664781 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 664783 T1917 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11cda57 name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 664783 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 664784 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 664787 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 664793 T1881 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 666145 T1881 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 666146 T1881 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 666155 T1881 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 666156 T1881 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:50909
   [junit4]   2> 666159 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 666160 T1881 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 666160 T1881 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003
   [junit4]   2> 666160 T1881 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003\'
   [junit4]   2> 666191 T1881 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003\solr.xml
   [junit4]   2> 666201 T1881 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003
   [junit4]   2> 666202 T1881 oasc.CoreContainer.<init> New CoreContainer 11913574
   [junit4]   2> 666202 T1881 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003\]
   [junit4]   2> 666203 T1881 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 666203 T1881 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 666203 T1881 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 666204 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 666204 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 666204 T1881 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 666205 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 666205 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 666205 T1881 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 666206 T1881 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 666209 T1881 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 666211 T1881 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 666211 T1881 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 666211 T1881 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 666212 T1881 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50887/solr
   [junit4]   2> 666212 T1881 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 666212 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 666213 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 666229 T1929 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@115393e name:ZooKeeperConnection Watcher:127.0.0.1:50887 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 666230 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 666230 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 666232 T1883 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x149ff62cdfc0006, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 666234 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 666240 T1932 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1fa8465 name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 666240 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 666250 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 667259 T1881 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50909_p_gg%2Fdv
   [junit4]   2> 667261 T1881 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50909_p_gg%2Fdv
   [junit4]   2> 667272 T1881 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 667277 T1881 oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003
   [junit4]   2> 667960 T1881 oasc.CorePropertiesLocator.discoverUnder Found core conf in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003\conf\
   [junit4]   2> 668075 T1881 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 668077 T1881 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 668077 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 669281 T1881 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 669281 T1881 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 669286 T1881 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 669288 T1881 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:50918
   [junit4]   2> 669292 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 669293 T1881 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 669293 T1881 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004
   [junit4]   2> 669293 T1881 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\'
   [junit4]   2> 669324 T1881 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\solr.xml
   [junit4]   2> 669335 T1881 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004
   [junit4]   2> 669336 T1881 oasc.CoreContainer.<init> New CoreContainer 19289259
   [junit4]   2> 669336 T1881 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\]
   [junit4]   2> 669337 T1881 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 669337 T1881 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 669337 T1881 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 669338 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 669338 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 669338 T1881 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 669338 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 669339 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 669339 T1881 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 669339 T1881 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 669343 T1881 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 669346 T1881 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 669347 T1881 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 669347 T1881 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 669347 T1881 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50887/solr
   [junit4]   2> 669348 T1881 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 669348 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 669351 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 669353 T1945 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@6ee49f name:ZooKeeperConnection Watcher:127.0.0.1:50887 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 669354 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 669354 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 669358 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 669359 T1883 oazs.NIOServerCnxn.doIO WARN caught end of stream exception EndOfStreamException: Unable to read additional data from client sessionid 0x149ff62cdfc0008, likely client has closed socket
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
   [junit4]   2> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 669362 T1948 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@67030b name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 669362 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 669374 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 670381 T1881 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50918_p_gg%2Fdv
   [junit4]   2> 670383 T1881 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50918_p_gg%2Fdv
   [junit4]   2> 670393 T1881 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 670398 T1881 oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004
   [junit4]   2> 670949 T1881 oasc.CorePropertiesLocator.discoverUnder Found core conf in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\conf\
   [junit4]   2> 671009 T1881 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 671011 T1881 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 671011 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 672200 T1881 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 3
   [junit4]   2> 672201 T1881 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 672207 T1881 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 672209 T1881 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:50927
   [junit4]   2> 672214 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 672214 T1881 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 672215 T1881 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005
   [junit4]   2> 672215 T1881 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005\'
   [junit4]   2> 672252 T1881 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005\solr.xml
   [junit4]   2> 672266 T1881 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005
   [junit4]   2> 672267 T1881 oasc.CoreContainer.<init> New CoreContainer 7043761
   [junit4]   2> 672267 T1881 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005\]
   [junit4]   2> 672268 T1881 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 672268 T1881 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 672268 T1881 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 672269 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 672269 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 672269 T1881 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 672270 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 672270 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 672270 T1881 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 672271 T1881 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 672280 T1881 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 672282 T1881 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 672283 T1881 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 672283 T1881 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 672284 T1881 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50887/solr
   [junit4]   2> 672284 T1881 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 672284 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 672286 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 672290 T1961 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c4587 name:ZooKeeperConnection Watcher:127.0.0.1:50887 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 672291 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 672291 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 672296 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 672300 T1964 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@dd9180 name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 672301 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 672314 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 673323 T1881 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50927_p_gg%2Fdv
   [junit4]   2> 673324 T1881 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50927_p_gg%2Fdv
   [junit4]   2> 673339 T1881 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 673344 T1881 oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005
   [junit4]   2> 673974 T1881 oasc.CorePropertiesLocator.discoverUnder Found core conf in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005\conf\
   [junit4]   2> 674141 T1881 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 674142 T1881 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 674143 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 675358 T1881 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 4
   [junit4]   2> 675358 T1881 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 675366 T1881 oejus.SslContextFactory.doStart Enabled Protocols [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2] of [SSLv2Hello, SSLv3, TLSv1, TLSv1.1, TLSv1.2]
   [junit4]   2> 675368 T1881 oejs.AbstractConnector.doStart Started SslSelectChannelConnector@127.0.0.1:50936
   [junit4]   2> 675373 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 675373 T1881 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 675373 T1881 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006
   [junit4]   2> 675373 T1881 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\'
   [junit4]   2> 675398 T1881 oasc.ConfigSolr.fromFile Loading container configuration from C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\solr.xml
   [junit4]   2> 675412 T1881 oasc.CorePropertiesLocator.<init> Config-defined core root directory: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006
   [junit4]   2> 675413 T1881 oasc.CoreContainer.<init> New CoreContainer 31822724
   [junit4]   2> 675413 T1881 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\]
   [junit4]   2> 675413 T1881 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 675414 T1881 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 675414 T1881 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 675414 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 675414 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 675414 T1881 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 675414 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 675414 T1881 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 675415 T1881 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 675415 T1881 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 675417 T1881 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 675419 T1881 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 675419 T1881 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 675419 T1881 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 675419 T1881 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:50887/solr
   [junit4]   2> 675420 T1881 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 675420 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 675421 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 675425 T1977 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@2092d5 name:ZooKeeperConnection Watcher:127.0.0.1:50887 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 675426 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 675426 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 675431 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 675433 T1980 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@110b7ca name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 675434 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 675444 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 676449 T1881 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:50936_p_gg%2Fdv
   [junit4]   2> 676452 T1881 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:50936_p_gg%2Fdv
   [junit4]   2> 676464 T1881 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 676472 T1881 oasc.CorePropertiesLocator.discover Looking for core definitions underneath C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006
   [junit4]   2> 677561 T1881 oasc.CorePropertiesLocator.discoverUnder Found core conf in C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\conf\
   [junit4]   2> 677778 T1881 oasc.CorePropertiesLocator.discover Found 1 core definitions
   [junit4]   2> 677780 T1881 oass.SolrDispatchFilter.init user.dir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0
   [junit4]   2> 677781 T1881 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 677787 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 677788 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 677791 T1984 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@87324c name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 677792 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 677792 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 677794 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 677852 T1902 oass.SolrDispatchFilter.handleAdminRequest [admin] webapp=null path=/admin/collections params={val=false&name=legacyCloud&action=clusterprop&wt=javabin&version=2} status=0 QTime=6 
   [junit4]   2> 677853 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 677859 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 0, message = {
   [junit4]   2> 	  "operation":"clusterprop",
   [junit4]   2> 	  "name":"legacyCloud",
   [junit4]   2> 	  "val":"false"}
   [junit4]   2> 677864 T1911 oasco.CollectionMutator.checkKeyExistence ERROR Skipping invalid Overseer message because it has no collection specified: {
   [junit4]   2> 	  "operation":"clusterprop",
   [junit4]   2> 	  "name":"legacyCloud",
   [junit4]   2> 	  "val":"false"}
   [junit4]   2> 677867 T1911 oasc.Overseer$ClusterStateUpdater.run ERROR Overseer could not process the current clusterstate state update message, skipping the message. org.apache.solr.common.SolrException: Overseer SETREPLICAPROPERTY requires collection and shard and replica and property and property.value no action taken.
   [junit4]   2> 	at org.apache.solr.cloud.overseer.ReplicaMutator.addReplicaProperty(ReplicaMutator.java:96)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.processMessage(Overseer.java:444)
   [junit4]   2> 	at org.apache.solr.cloud.Overseer$ClusterStateUpdater.run(Overseer.java:323)
   [junit4]   2> 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 
   [junit4]   2> 677870 T1881 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 677872 T1881 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 677876 T1987 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@129fc43 name:ZooKeeperConnection Watcher:127.0.0.1:50887/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 677876 T1881 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 677877 T1881 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 677879 T1881 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 677894 T1904 oasha.CollectionsHandler.handleCreateAction Creating Collection : replicationFactor=2&maxShardsPerNode=2&name=delDeadColl&action=CREATE&numShards=2&wt=javabin&version=2
   [junit4]   2> 677896 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/collection-queue-work state SyncConnected
   [junit4]   2> 677897 T1912 oasc.OverseerCollectionProcessor.run Overseer Collection Processor: Get the message id:/overseer/collection-queue-work/qn-0000000000 message:{
   [junit4]   2> 	  "operation":"create",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"delDeadColl",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "maxShardsPerNode":"2"}
   [junit4]   2> 677898 T1988 oasc.OverseerCollectionProcessor.processMessage WARN OverseerCollectionProcessor.processMessage : create , {
   [junit4]   2> 	  "operation":"create",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"delDeadColl",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "maxShardsPerNode":"2"}
   [junit4]   2> 677902 T1988 oasc.OverseerCollectionProcessor.createConfNode Only one config set found in zk - using it:conf1
   [junit4]   2> 677902 T1988 oasc.OverseerCollectionProcessor.createConfNode creating collections conf node /collections/delDeadColl 
   [junit4]   2> 677903 T1988 oascc.SolrZkClient.makePath makePath: /collections/delDeadColl
   [junit4]   2> 677908 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 677911 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"create",
   [junit4]   2> 	  "fromApi":"true",
   [junit4]   2> 	  "name":"delDeadColl",
   [junit4]   2> 	  "replicationFactor":"2",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "maxShardsPerNode":"2"}
   [junit4]   2> 677911 T1911 oasco.ClusterStateMutator.createCollection building a new cName: delDeadColl
   [junit4]   2> 677917 T1948 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 677918 T1910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 677918 T1964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 677919 T1980 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 677920 T1932 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 677920 T1984 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 677926 T1917 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 677926 T1987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678009 T1988 oasc.OverseerCollectionProcessor.createCollection Creating SolrCores for new collection delDeadColl, shardNames [shard1, shard2] , replicationFactor : 2
   [junit4]   2> 678009 T1988 oasc.OverseerCollectionProcessor.createCollection Creating shard delDeadColl_shard1_replica1 as part of slice shard1 of collection delDeadColl on 127.0.0.1:50909_p_gg%2Fdv
   [junit4]   2> 678013 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 678014 T1988 oasc.OverseerCollectionProcessor.createCollection Creating shard delDeadColl_shard1_replica2 as part of slice shard1 of collection delDeadColl on 127.0.0.1:50936_p_gg%2Fdv
   [junit4]   2> 678016 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50909/p_gg/dv"}
   [junit4]   2> 678016 T1911 oasco.SliceMutator.addReplica createReplica() {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50909/p_gg/dv"} 
   [junit4]   2> 678021 T1988 oasc.OverseerCollectionProcessor.createCollection Creating shard delDeadColl_shard2_replica1 as part of slice shard2 of collection delDeadColl on 127.0.0.1:50927_p_gg%2Fdv
   [junit4]   2> 678024 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50936/p_gg/dv"}
   [junit4]   2> 678025 T1911 oasco.SliceMutator.addReplica createReplica() {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50936/p_gg/dv"} 
   [junit4]   2> 678027 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 678027 T1988 oasc.OverseerCollectionProcessor.createCollection Creating shard delDeadColl_shard2_replica2 as part of slice shard2 of collection delDeadColl on 127.0.0.1:50918_p_gg%2Fdv
   [junit4]   2> 678031 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50927/p_gg/dv"}
   [junit4]   2> 678032 T1911 oasco.SliceMutator.addReplica createReplica() {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50927/p_gg/dv"} 
   [junit4]   2> 678033 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 678035 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50918/p_gg/dv"}
   [junit4]   2> 678036 T1911 oasco.SliceMutator.addReplica createReplica() {
   [junit4]   2> 	  "operation":"ADDREPLICA",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50918/p_gg/dv"} 
   [junit4]   2> 678039 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 678041 T1948 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678041 T1987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678041 T1917 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678041 T1964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678042 T1980 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678042 T1984 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678042 T1910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678042 T1932 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678197 T1974 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&coreNodeName=core_node2&collection.configName=conf1&name=delDeadColl_shard1_replica2&action=CREATE&numShards=2&collection=delDeadColl&shard=shard1&wt=javabin&version=2
   [junit4]   2> 678226 T1957 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&coreNodeName=core_node3&collection.configName=conf1&name=delDeadColl_shard2_replica1&action=CREATE&numShards=2&collection=delDeadColl&shard=shard2&wt=javabin&version=2
   [junit4]   2> 678239 T1940 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&coreNodeName=core_node4&collection.configName=conf1&name=delDeadColl_shard2_replica2&action=CREATE&numShards=2&collection=delDeadColl&shard=shard2&wt=javabin&version=2
   [junit4]   2> 678266 T1923 oasha.CoreAdminHandler.handleCreateAction core create command qt=/admin/cores&coreNodeName=core_node1&collection.configName=conf1&name=delDeadColl_shard1_replica1&action=CREATE&numShards=2&collection=delDeadColl&shard=shard1&wt=javabin&version=2
   [junit4]   2> 678342 T1974 oasc.ZkController.publish publishing core=delDeadColl_shard1_replica2 state=down collection=delDeadColl
   [junit4]   2> 678343 T1974 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delDeadColl_shard1_replica2
   [junit4]   2> 678344 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 678345 T1957 oasc.ZkController.publish publishing core=delDeadColl_shard2_replica1 state=down collection=delDeadColl
   [junit4]   2> 678346 T1957 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delDeadColl_shard2_replica1
   [junit4]   2> 678347 T1940 oasc.ZkController.publish publishing core=delDeadColl_shard2_replica2 state=down collection=delDeadColl
   [junit4]   2> 678347 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica2",
   [junit4]   2> 	  "core_node_name":"core_node2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50936/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50936_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678349 T1940 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delDeadColl_shard2_replica2
   [junit4]   2> 678349 T1911 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica2",
   [junit4]   2> 	  "core_node_name":"core_node2",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50936/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50936_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678355 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 2, message = {
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica1",
   [junit4]   2> 	  "core_node_name":"core_node3",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50927/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50927_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678358 T1911 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica1",
   [junit4]   2> 	  "core_node_name":"core_node3",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50927/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50927_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678362 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 678364 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 2, message = {
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica2",
   [junit4]   2> 	  "core_node_name":"core_node4",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50918/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50918_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678365 T1911 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"delDeadColl_shard2_replica2",
   [junit4]   2> 	  "core_node_name":"core_node4",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50918/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50918_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678370 T1923 oasc.ZkController.publish publishing core=delDeadColl_shard1_replica1 state=down collection=delDeadColl
   [junit4]   2> 678371 T1923 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for delDeadColl_shard1_replica1
   [junit4]   2> 678372 T1910 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 678373 T1911 oasc.Overseer$ClusterStateUpdater.processMessage processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50909/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50909_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678375 T1911 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"delDeadColl_shard1_replica1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"https://127.0.0.1:50909/p_gg/dv",
   [junit4]   2> 	  "node_name":"127.0.0.1:50909_p_gg%2Fdv",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"delDeadColl",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 678481 T1948 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678482 T1910 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678482 T1987 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678482 T1984 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678483 T1917 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678482 T1932 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678482 T1964 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 678482 T1980 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 5)
   [junit4]   2> 679344 T1974 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delDeadColl
   [junit4]   2> 679345 T1974 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 679345 T1974 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delDeadColl
   [junit4]   2> 679346 T1957 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delDeadColl
   [junit4]   2> 679347 T1957 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 679348 T1957 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delDeadColl
   [junit4]   2> 679348 T1974 oascc.ZkStateReader.readConfigName path=/collections/delDeadColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 679349 T1974 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\delDeadColl_shard1_replica2\'
   [junit4]   2> 679349 T1940 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delDeadColl
   [junit4]   2> 679352 T1940 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 679352 T1940 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delDeadColl
   [junit4]   2> 679353 T1957 oascc.ZkStateReader.readConfigName path=/collections/delDeadColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 679357 T1957 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-005\delDeadColl_shard2_replica1\'
   [junit4]   2> 679362 T1940 oascc.ZkStateReader.readConfigName path=/collections/delDeadColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 679362 T1940 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\delDeadColl_shard2_replica2\'
   [junit4]   2> 679373 T1923 oasc.ZkController.createCollectionZkNode Check for collection zkNode:delDeadColl
   [junit4]   2> 679379 T1923 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 679379 T1923 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delDeadColl
   [junit4]   2> 679383 T1923 oascc.ZkStateReader.readConfigName path=/collections/delDeadColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 679383 T1923 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-003\delDeadColl_shard1_replica1\'
   [junit4]   2> 679391 T1974 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 679410 T1940 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 679414 T1974 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 679419 T1923 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 679419 T1940 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 679425 T1957 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 679436 T1974 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 679439 T1974 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 679447 T1940 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 679449 T1940 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 679450 T1923 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 679451 T1957 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 679452 T1974 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 679455 T1974 oass.IndexSchema.readSchema [delDeadColl_shard1_replica2] Schema name=test
   [junit4]   2> 679470 T1940 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 679474 T1940 oass.IndexSchema.readSchema [delDeadColl_shard2_replica2] Schema name=test
   [junit4]   2> 679493 T1923 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 679499 T1923 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 679503 T1957 oasc.SolrConfig.<init> Using Lucene MatchVersion: 6.0.0
   [junit4]   2> 679507 T1957 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 679526 T1974 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 679538 T1974 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 679540 T1974 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 679543 T1923 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 679547 T1923 oass.IndexSchema.readSchema [delDeadColl_shard1_replica1] Schema name=test
   [junit4]   2> 679573 T1940 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 679576 T1940 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 679577 T1940 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 679580 T1957 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 679592 T1957 oass.IndexSchema.readSchema [delDeadColl_shard2_replica1] Schema name=test
   [junit4]   2> 679661 T1923 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 679663 T1957 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 679663 T1923 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 679664 T1923 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 679666 T1957 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 679667 T1957 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 679695 T1974 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679700 T1974 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679706 T1974 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 679709 T1940 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679710 T1974 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 679710 T1974 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 679710 T1974 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 679712 T1923 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679715 T1940 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679716 T1957 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679717 T1974 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 679717 T1974 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 679717 T1974 oasc.CoreContainer.create Creating SolrCore 'delDeadColl_shard1_replica2' using configuration from collection delDeadColl
   [junit4]   2> 679717 T1974 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 679718 T1974 oasc.SolrCore.<init> [delDeadColl_shard1_replica2] Opening new SolrCore at ..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\delDeadColl_shard1_replica2\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\delDeadColl_shard1_replica2\data\
   [junit4]   2> 679718 T1974 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c96c8
   [junit4]   2> 679719 T1974 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\delDeadColl_shard1_replica2\data\
   [junit4]   2> 679719 T1974 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\delDeadColl_shard1_replica2\data\index/
   [junit4]   2> 679719 T1974 oasc.SolrCore.initIndex WARN [delDeadColl_shard1_replica2] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\delDeadColl_shard1_replica2\data\index' doesn't exist. Creating new index...
   [junit4]   2> 679720 T1974 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\..\..\..\..\..\..\..\..\..\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-006\delDeadColl_shard1_replica2\data\index
   [junit4]   2> 679720 T1974 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=47, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.8867238024671883]
   [junit4]   2> 679722 T1974 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@19be8be lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@b83ff1),segFN=segments_1,generation=1}
   [junit4]   2> 679722 T1974 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 679723 T1974 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 679724 T1974 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 679724 T1974 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 679724 T1974 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 679725 T1974 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 679725 T1974 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 679725 T1974 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 679726 T1974 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 679726 T1974 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 679726 T1974 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 679727 T1974 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 679727 T1974 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 679727 T1974 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 679728 T1974 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
   [junit4]   2> 679728 T1974 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler
   [junit4]   2> 679729 T1974 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler
   [junit4]   2> 679729 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
   [junit4]   2> 679730 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
   [junit4]   2> 679730 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
   [junit4]   2> 679731 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
   [junit4]   2> 679731 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
   [junit4]   2> 679731 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler
   [junit4]   2> 679732 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
   [junit4]   2> 679732 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler
   [junit4]   2> 679732 T1974 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 679733 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 679733 T1974 oasc.RequestHandlers.initHandlersFromConfig created /admin/fileedit: solr.admin.EditFileRequestHandler
   [junit4]   2> 679738 T1974 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 679739 T1974 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 679740 T1974 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 679740 T1974 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 679744 T1974 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 679744 T1974 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 679745 T1974 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=4, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.45034392590936567]
   [junit4]   2> 679745 T1974 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@19be8be lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@b83ff1),segFN=segments_1,generation=1}
   [junit4]   2> 679745 T1974 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 679746 T1974 oass.SolrIndexSearcher.<init> Opening Searcher@b40dcf[delDeadColl_shard1_replica2] main
   [junit4]   2> 679746 T1974 oascc.ZkStateReader.readConfigName Load collection config from:/collections/delDeadColl
   [junit4]   2> 679749 T1923 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679752 T1940 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 679754 T1957 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 679756 T1974 oascc.ZkStateReader.readConfigName path=/collections/delDeadColl configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 679756 T1974 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 679758 T1923 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 679759 T1974 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 679759 T1974 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 679759 T1974 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 679760 T1940 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 679760 T1940 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 679760 T1940 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 679762 T1957 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 679763 T1974 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 679763 T1974 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 679765 T1923 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 679765 T1923 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 679765 T1923 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 679766 T1974 oascc.SolrZkClient.makePath makePath: /configs/conf1/_rest_managed.json
   [junit4]   2> 679767 T1940 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 679767 T1940 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 679767 T1940 oasc.CoreContainer.create Creating SolrCore 'delDeadColl_shard2_replica2' using configuration from collection delDeadColl
   [junit4]   2> 679767 T1940 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 679768 T1940 oasc.SolrCore.<init> [delDeadColl_shard2_replica2] Opening new SolrCore at C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\delDeadColl_shard2_replica2\, dataDir=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\delDeadColl_shard2_replica2\data\
   [junit4]   2> 679768 T1940 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@1c96c8
   [junit4]   2> 679768 T1940 oasc.CachingDirectoryFactory.get return new directory for C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\delDeadColl_shard2_replica2\data\
   [junit4]   2> 679768 T1940 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplicaTest-4F62D3D25588F392-001\tempDir-004\delDeadColl_shard2_replica2\data\index/
   [junit4]   2> 679768 T1940 oasc.SolrCore.initIndex WARN [delDeadColl_shard2_replica2] Solr index directory 'C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build\solr-core\test\J0\temp\solr.cloud.DeleteInactiveReplica

[...truncated too long message...]

Schema, CurrencyFieldXmlFileTest, TestQueryUtils, OverseerStatusTest, SpellPossibilityIteratorTest, ResponseLogComponentTest, TestOmitPositions, DistributedFacetPivotSmallAdvancedTest, MergeStrategyTest, TestLMJelinekMercerSimilarityFactory, TestInitQParser, DateRangeFieldTest, TestSweetSpotSimilarityFactory, TestFileDictionaryLookup, RemoteQueryErrorTest, CustomCollectionTest, TestLRUCache, SpatialFilterTest, DistributedSpellCheckComponentTest, ChangedSchemaMergeTest, BJQParserTest, ChaosMonkeySafeLeaderTest, DateMathParserTest, TestRealTimeGet, AddSchemaFieldsUpdateProcessorFactoryTest, TestDistributedSearch, TestDistributedMissingSort, HdfsBasicDistributedZkTest, SuggesterTSTTest, SolrIndexConfigTest, TestMissingGroups, TestUpdate, TestBM25SimilarityFactory, TestSolrQueryParserDefaultOperatorResource, TestXIncludeConfig, UniqFieldsUpdateProcessorFactoryTest, TestAddFieldRealTimeGet, TestCodecSupport, TestSolrXmlPersistor, FileBasedSpellCheckerTest, HdfsUnloadDistributedZkTest, TestFastOutputStream, TestCustomSort, TestSolr4Spatial2, TestAnalyzeInfixSuggestions, TestExpandComponent, TestSchemaManager, TestQuerySenderListener, AnalysisErrorHandlingTest, CloudExitableDirectoryReaderTest, TermVectorComponentTest, TestStressVersions, SliceStateTest, SolrCoreTest, MBeansHandlerTest, TestPivotHelperCode, LeaderFailoverAfterPartitionTest, HighlighterMaxOffsetTest, TestClassNameShortening, ResourceLoaderTest, TestSolrJ, TestSolrConfigHandler, PeerSyncTest, TestLuceneMatchVersion, TestWriterPerf, OpenExchangeRatesOrgProviderTest, TestIndexingPerformance, TestManagedStopFilterFactory, TestStressRecovery, TestDynamicFieldCollectionResource, PingRequestHandlerTest, TestStandardQParsers, AnalysisAfterCoreReloadTest, TestModifyConfFiles, TestSolrCoreProperties, TestQueryTypes, TriLevelCompositeIdRoutingTest, ConvertedLegacyTest, DistributedFacetPivotLargeTest, ScriptEngineTest, CoreAdminHandlerTest, DistributedIntervalFacetingTest, TestSystemIdResolver, TestTrie, PrimUtilsTest, DisMaxRequestHandlerTest, TestPerFieldSimilarity, AsyncMigrateRouteKeyTest, TestCollectionAPI, TestSolrQueryParser, ExternalCollectionsTest, ClusterStateTest, SampleTest, HdfsDirectoryTest, TestCopyFieldCollectionResource, SyncSliceTest, ZkCLITest, SuggesterTest, ExitableDirectoryReaderTest, BasicDistributedZk2Test, TestHighlightDedupGrouping, TestSortingResponseWriter, PathHierarchyTokenizerFactoryTest, ChaosMonkeyNothingIsSafeTest, DocValuesTest, TestAtomicUpdateErrorCases, FacetPivotSmallTest, TestQuerySenderNoQuery, CollectionsAPIDistributedZkTest, LeaderElectionIntegrationTest, TestConfigOverlay, TestConfigSets, AssignTest, SOLR749Test, ZkSolrClientTest, TestManagedSchemaFieldResource, TestWordDelimiterFilterFactory, StatsComponentTest, SimpleMLTQParserTest, EchoParamsTest, TestFiltering, TestFieldTypeCollectionResource, OverseerCollectionProcessorTest, TestComponentsName, TestMergePolicyConfig, UpdateParamsTest, OutOfBoxZkACLAndCredentialsProvidersTest, TestNRTOpen, HighlighterTest, TestLMDirichletSimilarityFactory, TestInfoStreamLogging, SignatureUpdateProcessorFactoryTest, TestZkChroot, DocumentBuilderTest, TestBulkSchemaConcurrent, HdfsWriteToMultipleCollectionsTest, TestShortCircuitedRequests, PluginInfoTest, HighlighterConfigTest, TestReplicaProperties, TestManagedResource, OutputWriterTest, UpdateRequestProcessorFactoryTest, TestSchemaVersionResource, SimpleFacetsTest, FieldAnalysisRequestHandlerTest, TestDocBasedVersionConstraints, TestCSVResponseWriter, SearchHandlerTest, NoCacheHeaderTest, TestNonNRTOpen, TestFaceting, JsonLoaderTest, SuggesterWFSTTest, ClusterStateUpdateTest, TestRequestStatusCollectionAPI, TestConfig, TestCoreDiscovery, TestBlendedInfixSuggestions, LoggingHandlerTest, TestFieldTypeResource, TestTolerantSearch, DistributedQueryElevationComponentTest, DirectSolrSpellCheckerTest, SSLMigrationTest, SpellCheckComponentTest, RegexBoostProcessorTest, CurrencyFieldOpenExchangeTest, ZkNodePropsTest, TestStressLucene, AddBlockUpdateTest, TestComplexPhraseQParserPlugin, HdfsRecoveryZkTest, TestRandomMergePolicy, TestUtils, TimeZoneUtilsTest, HdfsLockFactoryTest, SolrPluginUtilsTest, CoreAdminCreateDiscoverTest, SharedFSAutoReplicaFailoverUtilsTest, SystemInfoHandlerTest, TestSolrQueryParserResource, BasicFunctionalityTest, TestRestManager, BasicDistributedZkTest, UnloadDistributedZkTest, OpenCloseCoreStressTest, OverseerTest, ShardRoutingTest, BasicZkTest, RecoveryZkTest, TestReplicationHandler, TestRandomFaceting, LeaderElectionTest, TestRandomDVFaceting, AutoCommitTest, TestDistributedGrouping, TestRecovery, TestHashPartitioner, TermVectorComponentDistributedTest, TestStressReorder, TestJoin, TestMultiCoreConfBootstrap, TestReload, DistributedTermsComponentTest, TestRangeQuery, TestSolr4Spatial, TestGroupingSearch, SolrCmdDistributorTest, QueryElevationComponentTest, TestSort, TestFunctionQuery, TestLazyCores, DirectUpdateHandlerTest, TestBadConfig, TestIndexSearcher, SoftAutoCommitTest, ShowFileRequestHandlerTest, SolrIndexSplitterTest, SimplePostToolTest, SpellCheckCollatorTest, SuggesterFSTTest, SolrRequestParserTest, TestFoldingMultitermQuery, TestCSVLoader, PolyFieldTest, WordBreakSolrSpellCheckerTest, SchemaVersionSpecificBehaviorTest, SolrCoreCheckLockOnStartupTest, TestPseudoReturnFields, QueryEqualityTest, DirectUpdateHandlerOptimizeTest, DefaultValueUpdateProcessorTest, StatelessScriptUpdateProcessorFactoryTest, TestRemoteStreaming, DocValuesMultiTest, DistanceFunctionTest, TestSolrDeletionPolicy1, SolrInfoMBeanTest, XsltUpdateRequestHandlerTest, DebugComponentTest, CacheHeaderTest, IndexBasedSpellCheckerTest, TestSurroundQueryParser, TestReversedWildcardFilterFactory, DocumentAnalysisRequestHandlerTest, PrimitiveFieldTypeTest, TermsComponentTest, XmlUpdateRequestHandlerTest, RequestHandlersTest, TestValueSourceCache, MoreLikeThisHandlerTest, RequiredFieldsTest, FastVectorHighlighterTest, IndexSchemaRuntimeFieldTest, IndexSchemaTest, TestJmxIntegration, TestCollationField, ReturnFieldsTest, JSONWriterTest, QueryParsingTest, TestAnalyzedSuggestions, TestPartialUpdateDeduplication, TestPHPSerializedResponseWriter, CSVRequestHandlerTest, TestBinaryResponseWriter, TestLFUCache, AlternateDirectoryTest, TestSolrIndexConfig, CopyFieldTest, BadComponentTest, TestDocSet, TestSearchPerf, NumericFieldsTest, MinimalSchemaTest, TestElisionMultitermQuery, TestFuzzyAnalyzedSuggestions, TestPostingsSolrHighlighter, DirectSolrConnectionTest, NotRequiredUniqueKeyTest, TestPhraseSuggestions, TestCharFilters, SynonymTokenizerTest, TestDFRSimilarityFactory, TestIBSimilarityFactory, TestDefaultSimilarityFactory, TestFastWriter, LegacyHTMLStripCharFilterTest, TestJmxMonitoredMap, URLClassifyProcessorTest, TestFastLRUCache, PreAnalyzedFieldTest, TestSuggestSpellingConverter, DateFieldTest, SpellingQueryConverterTest, DOMUtilTest, RAMDirectoryFactoryTest, TestDocumentBuilder, UUIDFieldTest, FileUtilsTest, CircularListTest, CursorPagingTest, SolrTestCaseJ4Test, TestCursorMarkWithoutUniqueKey, AliasIntegrationTest, CollectionsAPIAsyncDistributedZkTest, DeleteReplicaTest, DeleteShardTest, DistribCursorPagingTest, DistribDocExpirationUpdateProcessorTest, DistributedQueueTest, HttpPartitionTest, LeaderInitiatedRecoveryOnCommitTest, MigrateRouteKeyTest, MultiThreadedOCPTest, OverriddenZkACLAndCredentialsProvidersTest, ReplicationFactorTest, RollingRestartTest, SharedFSAutoReplicaFailoverTest, SimpleCollectionCreateDeleteTest, SolrXmlInZkTest, TestCloudInspectUtil, TestCloudPivotFacet, TestDistribDocBasedVersion, TestLeaderElectionZkExpiry, VMParamsZkACLAndCredentialsProvidersTest, HdfsBasicDistributedZk2Test, HdfsChaosMonkeySafeLeaderTest, HdfsCollectionsAPIDistributedZkTest, HdfsSyncSliceTest, StressHdfsTest, TestClusterStateMutator, CachingDirectoryFactoryTest, QueryResultKeyTest, TestInitParams, TestReloadAndDeleteDocs, TestShardHandlerFactory, TestSolrDynamicMBean, TestSolrXml, TestSolrXmlPersistence, TestReplicationHandlerBackup]
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestReplicationHandlerBackup -Dtests.seed=4F62D3D25588F392 -Dtests.slow=true -Dtests.locale=es_MX -Dtests.timezone=HST -Dtests.asserts=false -Dtests.file.encoding=US-ASCII
   [junit4] ERROR   0.00s | TestReplicationHandlerBackup (suite) <<<
   [junit4]    > Throwable #1: java.lang.Exception: Suite timeout exceeded (>= 7200000 msec).
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([4F62D3D25588F392]:0)
   [junit4] Completed in 7221.83s, 1 test, 2 errors <<< FAILURES!

[...truncated 195 lines...]
BUILD FAILED
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:519: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:467: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:61: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:39: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\build.xml:189: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\solr\common-build.xml:510: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1349: The following error occurred while executing this line:
C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:956: There were test failures: 451 suites, 1840 tests, 3 suite-level errors, 1 error, 56 ignored (31 assumptions)

Total time: 257 minutes 56 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 32bit/jdk1.8.0_20 -server -XX:+UseParallelGC (asserts: false)
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



Re: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.8.0_40-ea-b09) - Build # 4460 - Still Failing!

Posted by Robert Muir <rc...@gmail.com>.
On Mon, Dec 1, 2014 at 12:24 PM, Chris Hostetter
<ho...@fucit.org> wrote:
>
> right, i guess a better way to restate my question -- now that i
> understand better what is being "mocked" -- is to ask:
>
> Shouldn't there an easy way (ie: sys prop?) for end users, who may
> primarily be interested in testing lucene/solr on *their* OS with *their*
> filesystem, to tell the test framework to skip all mocking of the
> filesystem?
>
> so if someone wants to try out a patch, or do regression testing before
> upgrading, and wants to beast all the lucene/solr tests on their
> fs/os/hardware stack can force all X runs to use their *real* filesystem
> w/o any mocking?
>

I dont think so.

Such a thing is unrelated to this change. We have had "act like
windows" and other filesystem-mocking code in our tests forever
(MockDirectoryWrapper), and we add lots of asserting-wrappers and
other mocks around the codec API and other key places during tests.

This change just allows us to mock filesystem behavior underneath the
Directory api, and its usable for testing things against filesystem
operations that don't go thru the directory api too.

The purpose of our tests is to ensure the code is correct and nothing more.

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.8.0_40-ea-b09) - Build # 4460 - Still Failing!

Posted by Chris Hostetter <ho...@fucit.org>.
: Its easiest to explain i think by pasting the code. for 10% of runs we
: don't wrap the filesystem at all. This is to ensure our "mock

Ah - ok.  the 10% use "real" filesystem is the biggest part i was missing.

So 90% of the time we wrap the filesystem in special checks; and 10% of 
the time when you do special checks we emulate windows *unless* we're 
already on windows.

: order to prevent that, we would have to do a whole lot more, including
: "mocking linux" on windows. But thats just a non-goal here. These
: mockfilesystems do not make it any better or any worse.

yeah -- i missunderstood the scope of what was being mocked.

: If you are running on windows, then your actual filesystem is already
: acting like windows. We don't need to add any code to accomplish
: "windows behavior" :)

right, i guess a better way to restate my question -- now that i 
understand better what is being "mocked" -- is to ask:

Shouldn't there an easy way (ie: sys prop?) for end users, who may 
primarily be interested in testing lucene/solr on *their* OS with *their* 
filesystem, to tell the test framework to skip all mocking of the 
filesystem?  

so if someone wants to try out a patch, or do regression testing before 
upgrading, and wants to beast all the lucene/solr tests on their 
fs/os/hardware stack can force all X runs to use their *real* filesystem 
w/o any mocking?

: 
: ---------------------------------------------------------------------
: To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
: For additional commands, e-mail: dev-help@lucene.apache.org
: 
: 

-Hoss
http://www.lucidworks.com/

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.8.0_40-ea-b09) - Build # 4460 - Still Failing!

Posted by Robert Muir <rc...@gmail.com>.
On Mon, Dec 1, 2014 at 11:46 AM, Chris Hostetter
<ho...@fucit.org> wrote:
>
> : windowsfs uses map<inode,integer> to determine if it can delete a
> : file, but this is troublesome on windows. One problem is that we cant
> : 100% compare files for equality without doing i/o. And doing this
> : itself can piss "the real windows" off. So I just disable windowsfs on
> : actual windows for now, i don't really have a way to test it and i
> : dont think it buys us anything right now to add lots of special code.
>
> I've got a massive backlog of lucene/solr related email, and haven't
> really been able to follow all the commits, so forgive me if this is a
> stupid question but...
>
> as i understand it, the test framework has a bunch of new stuff now for
> mocking the filesystem (cool!) and this is the context for the "windowsfs"
> refrence here (correct?) ... so my question is this:
>
> when saying "disable windowsfs on actual windows" does that mean:
>
> 1: (mock) windowsfs is removed from the pool of random mockfs's that are
> candidates for testing when we detect the OS is "windows"
>         ...or does that mean that...

Its easiest to explain i think by pasting the code. for 10% of runs we
don't wrap the filesystem at all. This is to ensure our "mock
filesystems" dont hide real bugs.
Otherwise, we add some minimal checks: we disable actual fsync calls
(its unnecessary), fail on leaked file handles, limit the number of
open files to 2048 (currently).
Then we occasionally act like windows, if we are on a non-windows machine.

// sometimes just use a bare filesystem
    if (random.nextInt(10) > 0) {
      fs = new DisableFsyncFS(fs).getFileSystem(null);
      fs = new LeakFS(fs).getFileSystem(null);
      fs = new HandleLimitFS(fs, MAX_OPEN_FILES).getFileSystem(null);
      // windows is currently slow
      if (random.nextInt(10) == 0) {
        // don't try to emulate windows on windows: they don't get along
        if (!Constants.WINDOWS) {
          fs = new WindowsFS(fs).getFileSystem(null);
        }
      }
    }

>
> The reason i ask: both approaches seem ot have serious downsides, and i'm
> trying to understand which devil we're choosing?
>
> the downside for #2 is that a test on linux which cliams it's using
> "(mock) windowsfs" might pass/fail diff then the same test with the same
> seed on an actual windows machine where the fs is really diff --- but that
> seems no worse then what we've had in the past before we made any attempt
> to mock out hte filesystem.

This is not something I am currently trying to address with tests. In
general, tests might fail on windows and pass on linux and so on. In
order to prevent that, we would have to do a whole lot more, including
"mocking linux" on windows. But thats just a non-goal here. These
mockfilesystems do not make it any better or any worse.

>
> the downside of #1 however seems to have a lot more serious impact on
> people who are actualy running windows, and really want to test "how does
> lucene/solr perform on windows" ... becuase they will *never* see their
> tests run on a windows filesystem (either mock or real)
>

If you are running on windows, then your actual filesystem is already
acting like windows. We don't need to add any code to accomplish
"windows behavior" :)

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.8.0_40-ea-b09) - Build # 4460 - Still Failing!

Posted by Chris Hostetter <ho...@fucit.org>.
: windowsfs uses map<inode,integer> to determine if it can delete a
: file, but this is troublesome on windows. One problem is that we cant
: 100% compare files for equality without doing i/o. And doing this
: itself can piss "the real windows" off. So I just disable windowsfs on
: actual windows for now, i don't really have a way to test it and i
: dont think it buys us anything right now to add lots of special code.

I've got a massive backlog of lucene/solr related email, and haven't 
really been able to follow all the commits, so forgive me if this is a 
stupid question but...

as i understand it, the test framework has a bunch of new stuff now for 
mocking the filesystem (cool!) and this is the context for the "windowsfs" 
refrence here (correct?) ... so my question is this:

when saying "disable windowsfs on actual windows" does that mean:

1: (mock) windowsfs is removed from the pool of random mockfs's that are 
candidates for testing when we detect the OS is "windows"
	...or does that mean that...
2: if the radomizer selects (mock) windowsfs *and* we detect the OS is 
windows, then we ignore the "(mock) windowsfs" and just use the real 
filesystem?

The reason i ask: both approaches seem ot have serious downsides, and i'm 
trying to understand which devil we're choosing?

the downside for #2 is that a test on linux which cliams it's using 
"(mock) windowsfs" might pass/fail diff then the same test with the same 
seed on an actual windows machine where the fs is really diff --- but that 
seems no worse then what we've had in the past before we made any attempt 
to mock out hte filesystem.

the downside of #1 however seems to have a lot more serious impact on 
people who are actualy running windows, and really want to test "how does 
lucene/solr perform on windows" ... becuase they will *never* see their 
tests run on a windows filesystem (either mock or real)


...or am i missunderstanding something fundemental?



: 
: 
: On Sun, Nov 30, 2014 at 5:31 AM, Michael McCandless
: <lu...@mikemccandless.com> wrote:
: > I think Rob fixed this with rev 1642486
: >
: > Mike McCandless
: >
: > http://blog.mikemccandless.com
: >
: >
: > On Sat, Nov 29, 2014 at 3:31 PM, Policeman Jenkins Server
: > <je...@thetaphi.de> wrote:
: >> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4460/
: >> Java: 64bit/jdk1.8.0_40-ea-b09 -XX:+UseCompressedOops -XX:+UseG1GC (asserts: true)
: >>
: >> 1 tests failed.
: >> FAILED:  org.apache.lucene.util.junitcompat.TestLeaveFilesIfTestFails.testWindowsUnremovableFile
: >>
: >> Error Message:
: >> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\core\test\J0\temp\lucene.util.junitcompat.TestLeaveFilesIfTestFails$Nested2-C5F2CA80130DA348-001\leftover-001\child.locked
: >>
: >> Stack Trace:
: >> java.nio.file.AccessDeniedException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\core\test\J0\temp\lucene.util.junitcompat.TestLeaveFilesIfTestFails$Nested2-C5F2CA80130DA348-001\leftover-001\child.locked
: >>         at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
: >>         at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
: >>         at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
: >>         at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
: >>         at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
: >>         at org.apache.lucene.mockfile.WindowsFS.getKey(WindowsFS.java:57)
: >>         at org.apache.lucene.mockfile.WindowsFS.onClose(WindowsFS.java:77)
: >>         at org.apache.lucene.mockfile.HandleTrackingFS$5.close(HandleTrackingFS.java:210)
: >>         at org.apache.lucene.util.junitcompat.TestLeaveFilesIfTestFails.testWindowsUnremovableFile(TestLeaveFilesIfTestFails.java:79)
: >>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
: >>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
: >>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
: >>         at java.lang.reflect.Method.invoke(Method.java:497)
: >>         at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
: >>         at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
: >>         at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
: >>         at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
: >>         at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
: >>         at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
: >>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
: >>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: >>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
: >>         at org.junit.rules.RunRules.evaluate(RunRules.java:18)
: >>         at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
: >>         at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
: >>         at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
: >>         at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
: >>         at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
: >>         at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
: >>         at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
: >>         at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
: >>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
: >>         at org.junit.rules.RunRules.evaluate(RunRules.java:18)
: >>         at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
: >>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:199)
: >>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:310)
: >>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
: >>
: >>
: >>
: >>
: >> Build Log:
: >> [...truncated 436 lines...]
: >>    [junit4] Suite: org.apache.lucene.util.junitcompat.TestLeaveFilesIfTestFails
: >>    [junit4]   1> sysout from nested test:
: >>    [junit4]   1>
: >>    [junit4]   1> syserr from nested test: NOTE: test params are: codec=Asserting(Lucene50): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=sr_CS, timezone=Etc/UCT
: >>    [junit4]   1> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.8.0_40-ea (64-bit)/cpus=2,threads=1,free=81523472,total=214958080
: >>    [junit4]   1> NOTE: All tests run in this JVM: [TestAutomaton, TestQueryBuilder, Test2BNumericDocValues, TestDirectory, TestDocumentsWriterDeleteQueue, TestBooleanOr, TestPositiveScoresOnlyCollector, TestMmapDirectory, TestBagOfPostings, TestOperations, TestFieldReuse, TestBooleanScorer, TestNoMergeScheduler, TestFieldMaskingSpanQuery, TestHighCompressionMode, TestFieldsReader, TestLucene50StoredFieldsFormat, TestFilterIterator, TestStressDeletes, TestIndexWriterOnDiskFull, TestForceMergeForever, MultiCollectorTest, TestNumericDocValuesUpdates, TestConsistentFieldNumbers, TestSimpleExplanationsOfNonMatches, TestPayloadExplanations, TestSegmentTermDocs, TestIsCurrent, TestNIOFSDirectory, TestMockAnalyzer, TestNumericRangeQuery32, TestMultiDocValues, TestLevenshteinAutomata, TestByteArrayDataInput, TestSortRandom, TestByteSlices, TestMultiMMap, Test2BTerms, TestIndexInput, TestDocValuesIndexing, TestDemoParallelLeafReader, TestAddIndexes, TestBoolean2, TestDemo, TestIndexWriter
 Exceptions2, TestManyFields, TestDocumentsWriterStallControl, TestRegExp, TestFieldValueFilter, TestLogMergePolicy, TestSearcherManager, Nested1, Nested2]
: >>    [junit4]   1> NOTE: reproduce with: ant test  -Dtestcase=Nested2 -Dtests.seed=C5F2CA80130DA348 -Dtests.slow=true -Dtests.locale=sr_CS -Dtests.timezone=Etc/UCT -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
: >>    [junit4]   1>
: >>    [junit4] ERROR   0.04s | TestLeaveFilesIfTestFails.testWindowsUnremovableFile <<<
: >>    [junit4]    > Throwable #1: java.nio.file.AccessDeniedException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\core\test\J0\temp\lucene.util.junitcompat.TestLeaveFilesIfTestFails$Nested2-C5F2CA80130DA348-001\leftover-001\child.locked
: >>    [junit4]    >        at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
: >>    [junit4]    >        at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
: >>    [junit4]    >        at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
: >>    [junit4]    >        at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
: >>    [junit4]    >        at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
: >>    [junit4]    >        at org.apache.lucene.mockfile.WindowsFS.getKey(WindowsFS.java:57)
: >>    [junit4]    >        at org.apache.lucene.mockfile.WindowsFS.onClose(WindowsFS.java:77)
: >>    [junit4]    >        at org.apache.lucene.mockfile.HandleTrackingFS$5.close(HandleTrackingFS.java:210)
: >>    [junit4]    >        at org.apache.lucene.util.junitcompat.TestLeaveFilesIfTestFails.testWindowsUnremovableFile(TestLeaveFilesIfTestFails.java:79)
: >>    [junit4] Completed in 0.10s, 2 tests, 1 error <<< FAILURES!
: >>
: >> [...truncated 1156 lines...]
: >> BUILD FAILED
: >> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:515: The following error occurred while executing this line:
: >> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:463: The following error occurred while executing this line:
: >> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:61: The following error occurred while executing this line:
: >> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:39: The following error occurred while executing this line:
: >> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build.xml:49: The following error occurred while executing this line:
: >> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1349: The following error occurred while executing this line:
: >> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:956: There were test failures: 405 suites, 3053 tests, 1 error, 63 ignored (53 assumptions)
: >>
: >> Total time: 10 minutes 43 seconds
: >> Build step 'Invoke Ant' marked build as failure
: >> [description-setter] Description set: Java: 64bit/jdk1.8.0_40-ea-b09 -XX:+UseCompressedOops -XX:+UseG1GC (asserts: true)
: >> Archiving artifacts
: >> Recording test results
: >> Email was triggered for: Failure - Any
: >> Sending email for trigger: Failure - Any
: >>
: >>
: >>
: >>
: >> ---------------------------------------------------------------------
: >> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
: >> For additional commands, e-mail: dev-help@lucene.apache.org
: >
: > ---------------------------------------------------------------------
: > To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
: > For additional commands, e-mail: dev-help@lucene.apache.org
: >
: 
: ---------------------------------------------------------------------
: To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
: For additional commands, e-mail: dev-help@lucene.apache.org
: 
: 

-Hoss
http://www.lucidworks.com/

Re: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.8.0_40-ea-b09) - Build # 4460 - Still Failing!

Posted by Robert Muir <rc...@gmail.com>.
Yes, sorry, my email was screwed up and my test fail responses were
going to a blackhole.

windowsfs uses map<inode,integer> to determine if it can delete a
file, but this is troublesome on windows. One problem is that we cant
100% compare files for equality without doing i/o. And doing this
itself can piss "the real windows" off. So I just disable windowsfs on
actual windows for now, i don't really have a way to test it and i
dont think it buys us anything right now to add lots of special code.


On Sun, Nov 30, 2014 at 5:31 AM, Michael McCandless
<lu...@mikemccandless.com> wrote:
> I think Rob fixed this with rev 1642486
>
> Mike McCandless
>
> http://blog.mikemccandless.com
>
>
> On Sat, Nov 29, 2014 at 3:31 PM, Policeman Jenkins Server
> <je...@thetaphi.de> wrote:
>> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4460/
>> Java: 64bit/jdk1.8.0_40-ea-b09 -XX:+UseCompressedOops -XX:+UseG1GC (asserts: true)
>>
>> 1 tests failed.
>> FAILED:  org.apache.lucene.util.junitcompat.TestLeaveFilesIfTestFails.testWindowsUnremovableFile
>>
>> Error Message:
>> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\core\test\J0\temp\lucene.util.junitcompat.TestLeaveFilesIfTestFails$Nested2-C5F2CA80130DA348-001\leftover-001\child.locked
>>
>> Stack Trace:
>> java.nio.file.AccessDeniedException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\core\test\J0\temp\lucene.util.junitcompat.TestLeaveFilesIfTestFails$Nested2-C5F2CA80130DA348-001\leftover-001\child.locked
>>         at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
>>         at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
>>         at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
>>         at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
>>         at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
>>         at org.apache.lucene.mockfile.WindowsFS.getKey(WindowsFS.java:57)
>>         at org.apache.lucene.mockfile.WindowsFS.onClose(WindowsFS.java:77)
>>         at org.apache.lucene.mockfile.HandleTrackingFS$5.close(HandleTrackingFS.java:210)
>>         at org.apache.lucene.util.junitcompat.TestLeaveFilesIfTestFails.testWindowsUnremovableFile(TestLeaveFilesIfTestFails.java:79)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>         at java.lang.reflect.Method.invoke(Method.java:497)
>>         at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
>>         at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>>         at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
>>         at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>>         at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
>>         at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
>>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
>>         at org.junit.rules.RunRules.evaluate(RunRules.java:18)
>>         at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
>>         at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
>>         at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
>>         at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>>         at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>>         at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>>         at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>>         at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>>         at org.junit.rules.RunRules.evaluate(RunRules.java:18)
>>         at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
>>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:199)
>>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:310)
>>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
>>
>>
>>
>>
>> Build Log:
>> [...truncated 436 lines...]
>>    [junit4] Suite: org.apache.lucene.util.junitcompat.TestLeaveFilesIfTestFails
>>    [junit4]   1> sysout from nested test:
>>    [junit4]   1>
>>    [junit4]   1> syserr from nested test: NOTE: test params are: codec=Asserting(Lucene50): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=sr_CS, timezone=Etc/UCT
>>    [junit4]   1> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.8.0_40-ea (64-bit)/cpus=2,threads=1,free=81523472,total=214958080
>>    [junit4]   1> NOTE: All tests run in this JVM: [TestAutomaton, TestQueryBuilder, Test2BNumericDocValues, TestDirectory, TestDocumentsWriterDeleteQueue, TestBooleanOr, TestPositiveScoresOnlyCollector, TestMmapDirectory, TestBagOfPostings, TestOperations, TestFieldReuse, TestBooleanScorer, TestNoMergeScheduler, TestFieldMaskingSpanQuery, TestHighCompressionMode, TestFieldsReader, TestLucene50StoredFieldsFormat, TestFilterIterator, TestStressDeletes, TestIndexWriterOnDiskFull, TestForceMergeForever, MultiCollectorTest, TestNumericDocValuesUpdates, TestConsistentFieldNumbers, TestSimpleExplanationsOfNonMatches, TestPayloadExplanations, TestSegmentTermDocs, TestIsCurrent, TestNIOFSDirectory, TestMockAnalyzer, TestNumericRangeQuery32, TestMultiDocValues, TestLevenshteinAutomata, TestByteArrayDataInput, TestSortRandom, TestByteSlices, TestMultiMMap, Test2BTerms, TestIndexInput, TestDocValuesIndexing, TestDemoParallelLeafReader, TestAddIndexes, TestBoolean2, TestDemo, TestIndexWriterExceptions2, TestManyFields, TestDocumentsWriterStallControl, TestRegExp, TestFieldValueFilter, TestLogMergePolicy, TestSearcherManager, Nested1, Nested2]
>>    [junit4]   1> NOTE: reproduce with: ant test  -Dtestcase=Nested2 -Dtests.seed=C5F2CA80130DA348 -Dtests.slow=true -Dtests.locale=sr_CS -Dtests.timezone=Etc/UCT -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
>>    [junit4]   1>
>>    [junit4] ERROR   0.04s | TestLeaveFilesIfTestFails.testWindowsUnremovableFile <<<
>>    [junit4]    > Throwable #1: java.nio.file.AccessDeniedException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\core\test\J0\temp\lucene.util.junitcompat.TestLeaveFilesIfTestFails$Nested2-C5F2CA80130DA348-001\leftover-001\child.locked
>>    [junit4]    >        at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
>>    [junit4]    >        at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
>>    [junit4]    >        at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
>>    [junit4]    >        at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
>>    [junit4]    >        at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
>>    [junit4]    >        at org.apache.lucene.mockfile.WindowsFS.getKey(WindowsFS.java:57)
>>    [junit4]    >        at org.apache.lucene.mockfile.WindowsFS.onClose(WindowsFS.java:77)
>>    [junit4]    >        at org.apache.lucene.mockfile.HandleTrackingFS$5.close(HandleTrackingFS.java:210)
>>    [junit4]    >        at org.apache.lucene.util.junitcompat.TestLeaveFilesIfTestFails.testWindowsUnremovableFile(TestLeaveFilesIfTestFails.java:79)
>>    [junit4] Completed in 0.10s, 2 tests, 1 error <<< FAILURES!
>>
>> [...truncated 1156 lines...]
>> BUILD FAILED
>> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:515: The following error occurred while executing this line:
>> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:463: The following error occurred while executing this line:
>> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:61: The following error occurred while executing this line:
>> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:39: The following error occurred while executing this line:
>> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build.xml:49: The following error occurred while executing this line:
>> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1349: The following error occurred while executing this line:
>> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:956: There were test failures: 405 suites, 3053 tests, 1 error, 63 ignored (53 assumptions)
>>
>> Total time: 10 minutes 43 seconds
>> Build step 'Invoke Ant' marked build as failure
>> [description-setter] Description set: Java: 64bit/jdk1.8.0_40-ea-b09 -XX:+UseCompressedOops -XX:+UseG1GC (asserts: true)
>> Archiving artifacts
>> Recording test results
>> Email was triggered for: Failure - Any
>> Sending email for trigger: Failure - Any
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Re: [JENKINS] Lucene-Solr-trunk-Windows (64bit/jdk1.8.0_40-ea-b09) - Build # 4460 - Still Failing!

Posted by Michael McCandless <lu...@mikemccandless.com>.
I think Rob fixed this with rev 1642486

Mike McCandless

http://blog.mikemccandless.com


On Sat, Nov 29, 2014 at 3:31 PM, Policeman Jenkins Server
<je...@thetaphi.de> wrote:
> Build: http://jenkins.thetaphi.de/job/Lucene-Solr-trunk-Windows/4460/
> Java: 64bit/jdk1.8.0_40-ea-b09 -XX:+UseCompressedOops -XX:+UseG1GC (asserts: true)
>
> 1 tests failed.
> FAILED:  org.apache.lucene.util.junitcompat.TestLeaveFilesIfTestFails.testWindowsUnremovableFile
>
> Error Message:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\core\test\J0\temp\lucene.util.junitcompat.TestLeaveFilesIfTestFails$Nested2-C5F2CA80130DA348-001\leftover-001\child.locked
>
> Stack Trace:
> java.nio.file.AccessDeniedException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\core\test\J0\temp\lucene.util.junitcompat.TestLeaveFilesIfTestFails$Nested2-C5F2CA80130DA348-001\leftover-001\child.locked
>         at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
>         at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
>         at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
>         at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
>         at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
>         at org.apache.lucene.mockfile.WindowsFS.getKey(WindowsFS.java:57)
>         at org.apache.lucene.mockfile.WindowsFS.onClose(WindowsFS.java:77)
>         at org.apache.lucene.mockfile.HandleTrackingFS$5.close(HandleTrackingFS.java:210)
>         at org.apache.lucene.util.junitcompat.TestLeaveFilesIfTestFails.testWindowsUnremovableFile(TestLeaveFilesIfTestFails.java:79)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:497)
>         at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
>         at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
>         at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
>         at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
>         at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
>         at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:30)
>         at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
>         at org.junit.rules.RunRules.evaluate(RunRules.java:18)
>         at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
>         at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
>         at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
>         at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
>         at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
>         at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
>         at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
>         at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
>         at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
>         at org.junit.rules.RunRules.evaluate(RunRules.java:18)
>         at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.execute(SlaveMain.java:199)
>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMain.main(SlaveMain.java:310)
>         at com.carrotsearch.ant.tasks.junit4.slave.SlaveMainSafe.main(SlaveMainSafe.java:12)
>
>
>
>
> Build Log:
> [...truncated 436 lines...]
>    [junit4] Suite: org.apache.lucene.util.junitcompat.TestLeaveFilesIfTestFails
>    [junit4]   1> sysout from nested test:
>    [junit4]   1>
>    [junit4]   1> syserr from nested test: NOTE: test params are: codec=Asserting(Lucene50): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=false,coord=no): {}, locale=sr_CS, timezone=Etc/UCT
>    [junit4]   1> NOTE: Windows 7 6.1 amd64/Oracle Corporation 1.8.0_40-ea (64-bit)/cpus=2,threads=1,free=81523472,total=214958080
>    [junit4]   1> NOTE: All tests run in this JVM: [TestAutomaton, TestQueryBuilder, Test2BNumericDocValues, TestDirectory, TestDocumentsWriterDeleteQueue, TestBooleanOr, TestPositiveScoresOnlyCollector, TestMmapDirectory, TestBagOfPostings, TestOperations, TestFieldReuse, TestBooleanScorer, TestNoMergeScheduler, TestFieldMaskingSpanQuery, TestHighCompressionMode, TestFieldsReader, TestLucene50StoredFieldsFormat, TestFilterIterator, TestStressDeletes, TestIndexWriterOnDiskFull, TestForceMergeForever, MultiCollectorTest, TestNumericDocValuesUpdates, TestConsistentFieldNumbers, TestSimpleExplanationsOfNonMatches, TestPayloadExplanations, TestSegmentTermDocs, TestIsCurrent, TestNIOFSDirectory, TestMockAnalyzer, TestNumericRangeQuery32, TestMultiDocValues, TestLevenshteinAutomata, TestByteArrayDataInput, TestSortRandom, TestByteSlices, TestMultiMMap, Test2BTerms, TestIndexInput, TestDocValuesIndexing, TestDemoParallelLeafReader, TestAddIndexes, TestBoolean2, TestDemo, TestIndexWriterExceptions2, TestManyFields, TestDocumentsWriterStallControl, TestRegExp, TestFieldValueFilter, TestLogMergePolicy, TestSearcherManager, Nested1, Nested2]
>    [junit4]   1> NOTE: reproduce with: ant test  -Dtestcase=Nested2 -Dtests.seed=C5F2CA80130DA348 -Dtests.slow=true -Dtests.locale=sr_CS -Dtests.timezone=Etc/UCT -Dtests.asserts=true -Dtests.file.encoding=ISO-8859-1
>    [junit4]   1>
>    [junit4] ERROR   0.04s | TestLeaveFilesIfTestFails.testWindowsUnremovableFile <<<
>    [junit4]    > Throwable #1: java.nio.file.AccessDeniedException: C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build\core\test\J0\temp\lucene.util.junitcompat.TestLeaveFilesIfTestFails$Nested2-C5F2CA80130DA348-001\leftover-001\child.locked
>    [junit4]    >        at sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:83)
>    [junit4]    >        at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:97)
>    [junit4]    >        at sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:102)
>    [junit4]    >        at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:53)
>    [junit4]    >        at sun.nio.fs.WindowsFileAttributeViews$Basic.readAttributes(WindowsFileAttributeViews.java:38)
>    [junit4]    >        at org.apache.lucene.mockfile.WindowsFS.getKey(WindowsFS.java:57)
>    [junit4]    >        at org.apache.lucene.mockfile.WindowsFS.onClose(WindowsFS.java:77)
>    [junit4]    >        at org.apache.lucene.mockfile.HandleTrackingFS$5.close(HandleTrackingFS.java:210)
>    [junit4]    >        at org.apache.lucene.util.junitcompat.TestLeaveFilesIfTestFails.testWindowsUnremovableFile(TestLeaveFilesIfTestFails.java:79)
>    [junit4] Completed in 0.10s, 2 tests, 1 error <<< FAILURES!
>
> [...truncated 1156 lines...]
> BUILD FAILED
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:515: The following error occurred while executing this line:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:463: The following error occurred while executing this line:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\build.xml:61: The following error occurred while executing this line:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\extra-targets.xml:39: The following error occurred while executing this line:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\build.xml:49: The following error occurred while executing this line:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:1349: The following error occurred while executing this line:
> C:\Users\JenkinsSlave\workspace\Lucene-Solr-trunk-Windows\lucene\common-build.xml:956: There were test failures: 405 suites, 3053 tests, 1 error, 63 ignored (53 assumptions)
>
> Total time: 10 minutes 43 seconds
> Build step 'Invoke Ant' marked build as failure
> [description-setter] Description set: Java: 64bit/jdk1.8.0_40-ea-b09 -XX:+UseCompressedOops -XX:+UseG1GC (asserts: true)
> Archiving artifacts
> Recording test results
> Email was triggered for: Failure - Any
> Sending email for trigger: Failure - Any
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org