You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2013/11/15 00:06:23 UTC

[JENKINS] Lucene-Solr-Tests-4.x-Java7 - Build # 1761 - Failure

Build: https://builds.apache.org/job/Lucene-Solr-Tests-4.x-Java7/1761/

2 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.solr.core.OpenCloseCoreStressTest

Error Message:
10 threads leaked from SUITE scope at org.apache.solr.core.OpenCloseCoreStressTest:     1) Thread[id=3375, name=commitScheduler-2489-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:724)    2) Thread[id=3347, name=commitScheduler-2470-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:724)    3) Thread[id=3380, name=commitScheduler-2497-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:724)    4) Thread[id=3379, name=commitScheduler-2495-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:724)    5) Thread[id=3345, name=commitScheduler-2476-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:724)    6) Thread[id=3349, name=commitScheduler-2479-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:724)    7) Thread[id=3372, name=commitScheduler-2485-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:724)    8) Thread[id=3376, name=commitScheduler-2491-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:724)    9) Thread[id=3344, name=commitScheduler-2475-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:724)   10) Thread[id=3346, name=commitScheduler-2473-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:724)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 10 threads leaked from SUITE scope at org.apache.solr.core.OpenCloseCoreStressTest: 
   1) Thread[id=3375, name=commitScheduler-2489-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
   2) Thread[id=3347, name=commitScheduler-2470-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
   3) Thread[id=3380, name=commitScheduler-2497-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
   4) Thread[id=3379, name=commitScheduler-2495-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
   5) Thread[id=3345, name=commitScheduler-2476-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
   6) Thread[id=3349, name=commitScheduler-2479-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
   7) Thread[id=3372, name=commitScheduler-2485-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
   8) Thread[id=3376, name=commitScheduler-2491-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
   9) Thread[id=3344, name=commitScheduler-2475-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
  10) Thread[id=3346, name=commitScheduler-2473-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
	at __randomizedtesting.SeedInfo.seed([3F6997CF3A2C204]:0)


FAILED:  junit.framework.TestSuite.org.apache.solr.core.OpenCloseCoreStressTest

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=3375, name=commitScheduler-2489-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:724)    2) Thread[id=3347, name=commitScheduler-2470-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:724)    3) Thread[id=3380, name=commitScheduler-2497-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:724)    4) Thread[id=3379, name=commitScheduler-2495-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:724)    5) Thread[id=3345, name=commitScheduler-2476-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:724)    6) Thread[id=3349, name=commitScheduler-2479-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:724)    7) Thread[id=3372, name=commitScheduler-2485-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:724)    8) Thread[id=3376, name=commitScheduler-2491-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:724)    9) Thread[id=3344, name=commitScheduler-2475-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:724)   10) Thread[id=3346, name=commitScheduler-2473-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)         at java.lang.Thread.run(Thread.java:724)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=3375, name=commitScheduler-2489-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
   2) Thread[id=3347, name=commitScheduler-2470-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
   3) Thread[id=3380, name=commitScheduler-2497-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
   4) Thread[id=3379, name=commitScheduler-2495-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
   5) Thread[id=3345, name=commitScheduler-2476-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
   6) Thread[id=3349, name=commitScheduler-2479-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
   7) Thread[id=3372, name=commitScheduler-2485-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
   8) Thread[id=3376, name=commitScheduler-2491-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
   9) Thread[id=3344, name=commitScheduler-2475-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
  10) Thread[id=3346, name=commitScheduler-2473-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:724)
	at __randomizedtesting.SeedInfo.seed([3F6997CF3A2C204]:0)




Build Log:
[...truncated 10487 lines...]
   [junit4] Suite: org.apache.solr.core.OpenCloseCoreStressTest
   [junit4] IGNOR/A 0.02s J0 | OpenCloseCoreStressTest.test10MinutesNew
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly)
   [junit4]   2> 1876652 T3316 oas.SolrTestCaseJ4.setUp ###Starting test15SecondsNew
   [junit4]   2> 1876684 T3316 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 1876689 T3316 oejs.AbstractConnector.doStart Started SocketConnector@127.0.0.1:60058
   [junit4]   2> 1876690 T3316 oass.SolrDispatchFilter.init SolrDispatchFilter.init()
   [junit4]   2> 1876690 T3316 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 1876690 T3316 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_
   [junit4]   2> 1876691 T3316 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/'
   [junit4]   2> 1876726 T3316 oasc.ConfigSolr.fromFile Loading container configuration from /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/solr.xml
   [junit4]   2> 1876801 T3316 oasc.ConfigSolrXml.<init> Config-defined core root directory: 
   [junit4]   2> 1876802 T3316 oasc.CoreContainer.<init> New CoreContainer 1233868313
   [junit4]   2> 1876803 T3316 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/]
   [junit4]   2> 1876805 T3316 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 1876805 T3316 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: http://
   [junit4]   2> 1876805 T3316 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 1876806 T3316 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 1876806 T3316 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 1876807 T3316 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 1876807 T3316 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 1876807 T3316 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 1876808 T3316 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 1876814 T3316 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 1876814 T3316 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 1876815 T3316 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 1876819 T3316 oasc.CorePropertiesLocator.discover Looking for core definitions underneath /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_
   [junit4]   2> 1876821 T3316 oasc.CorePropertiesLocator.discoverUnder Found core 00004_core in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00004_core/
   [junit4]   2> 1876822 T3316 oasc.CorePropertiesLocator.discoverUnder Found core 00000_core in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00000_core/
   [junit4]   2> 1876823 T3316 oasc.CorePropertiesLocator.discoverUnder Found core 00001_core in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00001_core/
   [junit4]   2> 1876825 T3316 oasc.CorePropertiesLocator.discoverUnder Found core 00003_core in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00003_core/
   [junit4]   2> 1876827 T3316 oasc.CorePropertiesLocator.discoverUnder Found core 00002_core in /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00002_core/
   [junit4]   2> 1876827 T3316 oasc.CorePropertiesLocator.discover Found 5 core definitions
   [junit4]   2> 1876828 T3316 oass.SolrDispatchFilter.init user.dir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0
   [junit4]   2> 1876829 T3316 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 1876873 T3316 oasc.OpenCloseCoreStressTest.doStress Starting indexing and querying
   [junit4]   2> 1876875 T3316 oasc.OpenCloseCoreStressTest.doStress 
   [junit4]   2> 	
   [junit4]   2> 	
   [junit4]   2> 	
   [junit4]   2> 	
   [junit4]   2> 	Starting a 15 second cycle, seconds left: 15. Seconds run so far: 0.
   [junit4]   2> 1876878 T3330 oasc.OneIndexer.run Starting indexing thread: 3330
   [junit4]   2> 1876878 T3328 oasc.OneIndexer.run Starting indexing thread: 3328
   [junit4]   2> 1876878 T3329 oasc.OneIndexer.run Starting indexing thread: 3329
   [junit4]   2> 1876878 T3331 oasc.OneIndexer.run Starting indexing thread: 3331
   [junit4]   2> 1876878 T3327 oasc.OneIndexer.run Starting indexing thread: 3327
   [junit4]   2> 1876886 T3333 oasc.OneQuery.run Starting query thread: 3333
   [junit4]   2> 1876886 T3320 oasc.CoreContainer.create Creating SolrCore '00004_core' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00004_core
   [junit4]   2> 1876886 T3336 oasc.OneQuery.run Starting query thread: 3336
   [junit4]   2> 1876886 T3335 oasc.OneQuery.run Starting query thread: 3335
   [junit4]   2> 1876886 T3334 oasc.OneQuery.run Starting query thread: 3334
   [junit4]   2> 1876886 T3319 oasc.CoreContainer.create Creating SolrCore '00002_core' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00002_core
   [junit4]   2> 1876886 T3332 oasc.OneQuery.run Starting query thread: 3332
   [junit4]   2> 1876888 T3319 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00002_core/'
   [junit4]   2> 1876887 T3320 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00004_core/'
   [junit4]   2> 1876887 T3322 oasc.CoreContainer.create Creating SolrCore '00003_core' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00003_core
   [junit4]   2> 1876886 T3321 oasc.CoreContainer.create Creating SolrCore '00000_core' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00000_core
   [junit4]   2> 1876890 T3322 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00003_core/'
   [junit4]   2> 1876890 T3321 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00000_core/'
   [junit4]   2> 1876954 T3319 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 1876959 T3321 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 1876965 T3320 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 1876970 T3322 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 1877027 T3319 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-minimal.xml
   [junit4]   2> 1877028 T3319 oass.IndexSchema.readSchema Reading Solr Schema from schema-tiny.xml
   [junit4]   2> 1877033 T3321 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-minimal.xml
   [junit4]   2> 1877034 T3321 oass.IndexSchema.readSchema Reading Solr Schema from schema-tiny.xml
   [junit4]   2> 1877036 T3319 oass.IndexSchema.readSchema [00002_core] Schema name=tiny
   [junit4]   2> 1877038 T3320 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-minimal.xml
   [junit4]   2> 1877039 T3320 oass.IndexSchema.readSchema Reading Solr Schema from schema-tiny.xml
   [junit4]   2> 1877042 T3321 oass.IndexSchema.readSchema [00000_core] Schema name=tiny
   [junit4]   2> 1877042 T3322 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-minimal.xml
   [junit4]   2> 1877043 T3322 oass.IndexSchema.readSchema Reading Solr Schema from schema-tiny.xml
   [junit4]   2> 1877046 T3320 oass.IndexSchema.readSchema [00004_core] Schema name=tiny
   [junit4]   2> 1877051 T3322 oass.IndexSchema.readSchema [00003_core] Schema name=tiny
   [junit4]   2> 1877071 T3319 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1877071 T3321 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1877073 T3319 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1877074 T3321 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1877074 T3319 oasc.SolrCore.<init> [00002_core] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00002_core/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00002_core/data/
   [junit4]   2> 1877074 T3321 oasc.SolrCore.<init> [00000_core] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00000_core/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00000_core/data/
   [junit4]   2> 1877075 T3321 oasc.SolrCore.<init> JMX monitoring not detected for core: 00000_core
   [junit4]   2> 1877075 T3320 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1877074 T3319 oasc.SolrCore.<init> JMX monitoring not detected for core: 00002_core
   [junit4]   2> 1877076 T3321 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00000_core/data
   [junit4]   2> 1877077 T3321 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00000_core/data/index/
   [junit4]   2> 1877077 T3320 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1877077 T3319 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00002_core/data
   [junit4]   2> 1877078 T3320 oasc.SolrCore.<init> [00004_core] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00004_core/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00004_core/data/
   [junit4]   2> 1877078 T3321 oasc.SolrCore.initIndex WARN [00000_core] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00000_core/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1877080 T3322 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1877080 T3321 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00000_core/data/index
   [junit4]   2> 1877079 T3320 oasc.SolrCore.<init> JMX monitoring not detected for core: 00004_core
   [junit4]   2> 1877079 T3319 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00002_core/data/index/
   [junit4]   2> 1877081 T3321 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@352aefe3
   [junit4]   2> 1877082 T3320 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00004_core/data
   [junit4]   2> 1877082 T3322 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1877082 T3319 oasc.SolrCore.initIndex WARN [00002_core] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00002_core/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1877084 T3322 oasc.SolrCore.<init> [00003_core] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00003_core/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00003_core/data/
   [junit4]   2> 1877085 T3322 oasc.SolrCore.<init> JMX monitoring not detected for core: 00003_core
   [junit4]   2> 1877083 T3321 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@e8588ac lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@44f166d2),segFN=segments_1,generation=1}
   [junit4]   2> 1877083 T3320 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00004_core/data/index/
   [junit4]   2> 1877086 T3322 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00003_core/data
   [junit4]   2> 1877086 T3321 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1877084 T3319 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00002_core/data/index
   [junit4]   2> 1877087 T3322 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00003_core/data/index/
   [junit4]   2> 1877086 T3320 oasc.SolrCore.initIndex WARN [00004_core] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00004_core/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1877089 T3321 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1877089 T3322 oasc.SolrCore.initIndex WARN [00003_core] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00003_core/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1877088 T3319 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@5bf63a91
   [junit4]   2> 1877091 T3322 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00003_core/data/index
   [junit4]   2> 1877090 T3321 oasc.RequestHandlers.initHandlersFromConfig created /select: solr.SearchHandler
   [junit4]   2> 1877090 T3320 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00004_core/data/index
   [junit4]   2> 1877092 T3321 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 1877092 T3322 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@4fd3a691
   [junit4]   2> 1877092 T3319 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@66de51ba lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@b0c98b8),segFN=segments_1,generation=1}
   [junit4]   2> 1877094 T3322 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@3b666155 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@43d2a2a2),segFN=segments_1,generation=1}
   [junit4]   2> 1877094 T3319 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1877093 T3321 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 1877093 T3320 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@71b2b7ef
   [junit4]   2> 1877096 T3319 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1877095 T3322 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1877097 T3319 oasc.RequestHandlers.initHandlersFromConfig created /select: solr.SearchHandler
   [junit4]   2> 1877097 T3320 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@27f34c2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@dd70d40),segFN=segments_1,generation=1}
   [junit4]   2> 1877098 T3322 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1877098 T3320 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1877098 T3319 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 1877099 T3322 oasc.RequestHandlers.initHandlersFromConfig created /select: solr.SearchHandler
   [junit4]   2> 1877100 T3319 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 1877100 T3320 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1877100 T3322 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 1877101 T3320 oasc.RequestHandlers.initHandlersFromConfig created /select: solr.SearchHandler
   [junit4]   2> 1877101 T3322 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 1877102 T3320 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 1877103 T3320 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 1877123 T3319 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1877124 T3321 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1877124 T3322 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1877124 T3320 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1877127 T3319 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1877128 T3319 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1877128 T3321 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1877129 T3322 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1877129 T3319 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=754840119, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1877129 T3322 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1877129 T3320 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1877129 T3321 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1877131 T3320 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1877131 T3322 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=754840119, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1877130 T3319 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@66de51ba lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@b0c98b8),segFN=segments_1,generation=1}
   [junit4]   2> 1877133 T3322 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@3b666155 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@43d2a2a2),segFN=segments_1,generation=1}
   [junit4]   2> 1877132 T3320 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=754840119, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1877132 T3321 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.AlcoholicMergePolicy: [AlcoholicMergePolicy: minMergeSize=0, mergeFactor=10, maxMergeSize=754840119, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.1]
   [junit4]   2> 1877135 T3320 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@27f34c2 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@dd70d40),segFN=segments_1,generation=1}
   [junit4]   2> 1877134 T3322 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1877133 T3319 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1877136 T3322 oass.SolrIndexSearcher.<init> Opening Searcher@10746814 main
   [junit4]   2> 1877136 T3320 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1877135 T3321 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@e8588ac lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@44f166d2),segFN=segments_1,generation=1}
   [junit4]   2> 1877138 T3322 oasc.SolrCores.putTransientCore Opening transient core 00003_core
   [junit4]   2> 1877138 T3342 oasc.SolrCore.registerSearcher [00003_core] Registered new searcher Searcher@10746814 main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 1877138 T3320 oass.SolrIndexSearcher.<init> Opening Searcher@4cd4ec79 main
   [junit4]   2> 1877137 T3319 oass.SolrIndexSearcher.<init> Opening Searcher@200c9a08 main
   [junit4]   2> 1877138 T3322 oasc.CoreContainer.registerCore registering core: 00003_core
   [junit4]   2> 1877140 T3343 oasc.SolrCore.registerSearcher [00004_core] Registered new searcher Searcher@4cd4ec79 main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 1877138 T3321 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1877140 T3340 oasc.SolrCore.registerSearcher [00002_core] Registered new searcher Searcher@200c9a08 main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 1877140 T3319 oasc.SolrCores.putTransientCore Opening transient core 00002_core
   [junit4]   2> 1877140 T3320 oasc.SolrCores.putTransientCore Opening transient core 00004_core
   [junit4]   2> 1877143 T3319 oasc.CoreContainer.registerCore registering core: 00002_core
   [junit4]   2> 1877141 T3321 oass.SolrIndexSearcher.<init> Opening Searcher@701502 main
   [junit4]   2> 1877143 T3320 oasc.CoreContainer.registerCore registering core: 00004_core
   [junit4]   2> 1877145 T3321 oasc.SolrCores.putTransientCore Opening transient core 00000_core
   [junit4]   2> 1877145 T3341 oasc.SolrCore.registerSearcher [00000_core] Registered new searcher Searcher@701502 main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 1877146 T3321 oasc.CoreContainer.registerCore registering core: 00000_core
   [junit4]   2> ASYNC  NEW_CORE C976 name=00003_core org.apache.solr.core.SolrCore@1b55a948
   [junit4]   2> 1877154 T3339 C976 oasc.SolrCore.execute [00003_core] webapp=/solr path=/select params={q=*:*&wt=javabin&qt=/select&version=2} hits=0 status=0 QTime=2 
   [junit4]   2> ASYNC  NEW_CORE C977 name=00002_core org.apache.solr.core.SolrCore@32da8e79
   [junit4]   2> 1877155 T3325 C977 oasc.SolrCore.execute [00002_core] webapp=/solr path=/select params={q=*:*&wt=javabin&qt=/select&version=2} hits=0 status=0 QTime=0 
   [junit4]   2> 1877156 T3322 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id4]} 0 4
   [junit4]   2> ASYNC  NEW_CORE C978 name=00004_core org.apache.solr.core.SolrCore@6afbb755
   [junit4]   2> 1877157 T3338 C978 oasc.SolrCore.execute [00004_core] webapp=/solr path=/select params={q=*:*&wt=javabin&qt=/select&version=2} hits=0 status=0 QTime=1 
   [junit4]   2> 1877157 T3323 C978 oasup.LogUpdateProcessor.finish [00004_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id5]} 0 2
   [junit4]   2> 1877158 T3324 C977 oasc.SolrCore.execute [00002_core] webapp=/solr path=/select params={q=*:*&wt=javabin&qt=/select&version=2} hits=0 status=0 QTime=1 
   [junit4]   2> 1877158 T3337 C978 oasc.SolrCore.execute [00004_core] webapp=/solr path=/select params={q=*:*&wt=javabin&qt=/select&version=2} hits=0 status=0 QTime=0 
   [junit4]   2> ASYNC  NEW_CORE C979 name=00000_core org.apache.solr.core.SolrCore@4e44a555
   [junit4]   2> 1877159 T3321 C979 oasup.LogUpdateProcessor.finish [00000_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id1]} 0 2
   [junit4]   2> 1877159 T3319 C977 oasup.LogUpdateProcessor.finish [00002_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id3]} 0 1
   [junit4]   2> 1877161 T3322 C977 oasup.LogUpdateProcessor.finish [00002_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id6]} 0 2
   [junit4]   2> 1877161 T3320 C978 oasup.LogUpdateProcessor.finish [00004_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id2]} 0 3
   [junit4]   2> 1877163 T3323 C979 oasup.LogUpdateProcessor.finish [00000_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id7]} 0 2
   [junit4]   2> 1877164 T3321 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id8]} 0 1
   [junit4]   2> 1877165 T3322 C979 oasup.LogUpdateProcessor.finish [00000_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id10]} 0 1
   [junit4]   2> 1877165 T3319 C977 oasup.LogUpdateProcessor.finish [00002_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id9]} 0 1
   [junit4]   2> 1877166 T3320 C979 oasup.LogUpdateProcessor.finish [00000_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id11]} 0 1
   [junit4]   2> 1877166 T3323 C979 oasup.LogUpdateProcessor.finish [00000_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id12]} 0 1
   [junit4]   2> 1877168 T3321 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id13]} 0 1
   [junit4]   2> 1877168 T3322 oasc.CoreContainer.create Creating SolrCore '00001_core' using instanceDir: /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00001_core
   [junit4]   2> 1877169 T3322 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00001_core/'
   [junit4]   2> 1877169 T3319 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id15]} 0 1
   [junit4]   2> 1877170 T3323 C977 oasup.LogUpdateProcessor.finish [00002_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id17]} 0 1
   [junit4]   2> 1877173 T3319 C978 oasup.LogUpdateProcessor.finish [00004_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id19]} 0 1
   [junit4]   2> 1877174 T3323 C978 oasup.LogUpdateProcessor.finish [00004_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id20]} 0 1
   [junit4]   2> 1877177 T3319 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id21]} 0 1
   [junit4]   2> 1877178 T3323 C977 oasup.LogUpdateProcessor.finish [00002_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id22]} 0 1
   [junit4]   2> 1877181 T3319 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id23]} 0 1
   [junit4]   2> 1877182 T3323 C979 oasup.LogUpdateProcessor.finish [00000_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id24]} 0 1
   [junit4]   2> 1877184 T3319 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id25]} 0 1
   [junit4]   2> 1877186 T3323 C977 oasup.LogUpdateProcessor.finish [00002_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id26]} 0 1
   [junit4]   2> 1877188 T3319 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id27]} 0 1
   [junit4]   2> 1877190 T3323 C978 oasup.LogUpdateProcessor.finish [00004_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id28]} 0 1
   [junit4]   2> 1877192 T3319 C978 oasup.LogUpdateProcessor.finish [00004_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id29]} 0 1
   [junit4]   2> 1877193 T3323 C977 oasup.LogUpdateProcessor.finish [00002_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id30]} 0 1
   [junit4]   2> 1877195 T3319 C979 oasup.LogUpdateProcessor.finish [00000_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id31]} 0 1
   [junit4]   2> 1877196 T3323 C978 oasup.LogUpdateProcessor.finish [00004_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id32]} 0 1
   [junit4]   2> 1877198 T3319 C979 oasup.LogUpdateProcessor.finish [00000_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id33]} 0 1
   [junit4]   2> 1877199 T3323 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id34]} 0 1
   [junit4]   2> 1877201 T3319 C977 oasup.LogUpdateProcessor.finish [00002_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id35]} 0 1
   [junit4]   2> 1877202 T3323 C979 oasup.LogUpdateProcessor.finish [00000_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id36]} 0 0
   [junit4]   2> 1877229 T3322 oasc.SolrConfig.<init> Using Lucene MatchVersion: LUCENE_47
   [junit4]   2> 1877262 T3325 C977 oasc.SolrCore.execute [00002_core] webapp=/solr path=/select params={q=*:*&wt=javabin&qt=/select&version=2} hits=0 status=0 QTime=0 
   [junit4]   2> 1877263 T3338 C977 oasc.SolrCore.execute [00002_core] webapp=/solr path=/select params={q=*:*&wt=javabin&qt=/select&version=2} hits=0 status=0 QTime=0 
   [junit4]   2> 1877264 T3324 C977 oasc.SolrCore.execute [00002_core] webapp=/solr path=/select params={q=*:*&wt=javabin&qt=/select&version=2} hits=0 status=0 QTime=0 
   [junit4]   2> 1877291 T3322 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig-minimal.xml
   [junit4]   2> 1877292 T3322 oass.IndexSchema.readSchema Reading Solr Schema from schema-tiny.xml
   [junit4]   2> 1877299 T3322 oass.IndexSchema.readSchema [00001_core] Schema name=tiny
   [junit4]   2> 1877325 T3322 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 1877327 T3322 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 1877328 T3322 oasc.SolrCore.<init> [00001_core] Opening new SolrCore at /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00001_core/, dataDir=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00001_core/data/
   [junit4]   2> 1877328 T3322 oasc.SolrCore.<init> JMX monitoring not detected for core: 00001_core
   [junit4]   2> 1877329 T3322 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00001_core/data
   [junit4]   2> 1877330 T3322 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00001_core/data/index/
   [junit4]   2> 1877330 T3322 oasc.SolrCore.initIndex WARN [00001_core] Solr index directory '/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00001_core/data/index' doesn't exist. Creating new index...
   [junit4]   2> 1877331 T3322 oasc.CachingDirectoryFactory.get return new directory for /usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build/solr-core/test/J0/./OpenCloseCoreStressTest_/00001_core/data/index
   [junit4]   2> 1877332 T3322 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=10, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.41045825606928077]
   [junit4]   2> 1877333 T3322 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@53e08a1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2e6e88b6),segFN=segments_1,generation=1}
   [junit4]   2> 1877334 T3322 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1877335 T3322 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 1877336 T3322 oasc.RequestHandlers.initHandlersFromConfig created /select: solr.SearchHandler
   [junit4]   2> 1877336 T3322 oasc.RequestHandlers.initHandlersFromConfig created /admin/: org.apache.solr.handler.admin.AdminHandlers
   [junit4]   2> 1877337 T3322 oasc.RequestHandlers.initHandlersFromConfig created /update: solr.UpdateRequestHandler
   [junit4]   2> 1877339 T3322 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 1877343 T3322 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 1877344 T3322 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 1877344 T3322 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=33, maxMergeAtOnceExplicit=44, maxMergedSegmentMB=27.138671875, floorSegmentMB=1.849609375, forceMergeDeletesPctAllowed=22.854052058908806, segmentsPerTier=50.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.4206029088279891
   [junit4]   2> 1877345 T3322 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(org.apache.lucene.store.RAMDirectory@53e08a1 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@2e6e88b6),segFN=segments_1,generation=1}
   [junit4]   2> 1877346 T3322 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 1877347 T3322 oass.SolrIndexSearcher.<init> Opening Searcher@545d511d main
   [junit4]   2> 1877348 T3322 oasc.SolrCores.putTransientCore Opening transient core 00001_core
   [junit4]   2> 1877348 T3348 oasc.SolrCore.registerSearcher [00001_core] Registered new searcher Searcher@545d511d main{StandardDirectoryReader(segments_1:1:nrt)}
   [junit4]   2> 1877348 T3322 oasc.CoreContainer.registerCore registering core: 00001_core
   [junit4]   2> ASYNC  NEW_CORE C980 name=00001_core org.apache.solr.core.SolrCore@2a727cf
   [junit4]   2> 1877359 T3337 C980 oasc.SolrCore.execute [00001_core] webapp=/solr path=/select params={q=*:*&wt=javabin&qt=/select&version=2} hits=0 status=0 QTime=0 
   [junit4]   2> 1877363 T3339 C980 oasc.SolrCore.execute [00001_core] webapp=/solr path=/select params={q=*:*&wt=javabin&qt=/select&version=2} hits=0 status=0 QTime=0 
   [junit4]   2> 1877364 T3321 C980 oasup.LogUpdateProcessor.finish [00001_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id18]} 0 5
   [junit4]   2> 1877364 T3322 C980 oasup.LogUpdateProcessor.finish [00001_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id14]} 0 5
   [junit4]   2> 1877364 T3323 C980 oasup.LogUpdateProcessor.finish [00001_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id38]} 0 4
   [junit4]   2> 1877366 T3319 C980 oasup.LogUpdateProcessor.finish [00001_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id37]} 0 1
   [junit4]   2> 1877366 T3325 C978 oasc.SolrCore.execute [00004_core] webapp=/solr path=/select params={q=*:*&wt=javabin&qt=/select&version=2} hits=0 status=0 QTime=0 
   [junit4]   2> 1877366 T3320 C980 oasup.LogUpdateProcessor.finish [00001_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id16]} 0 3
   [junit4]   2> 1877367 T3338 C979 oasc.SolrCore.execute [00000_core] webapp=/solr path=/select params={q=*:*&wt=javabin&qt=/select&version=2} hits=0 status=0 QTime=0 
   [junit4]   2> 1877367 T3324 C979 oasc.SolrCore.execute [00000_core] webapp=/solr path=/select params={q=*:*&wt=javabin&qt=/select&version=2} hits=0 status=0 QTime=0 
   [junit4]   2> 1877369 T3321 C980 oasup.LogUpdateProcessor.finish [00001_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id39]} 0 1
   [junit4]   2> 1877370 T3323 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id41]} 0 1
   [junit4]   2> 1877371 T3319 C980 oasup.LogUpdateProcessor.finish [00001_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id42]} 0 2
   [junit4]   2> 1877370 T3322 C978 oasup.LogUpdateProcessor.finish [00004_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id40]} 0 2
   [junit4]   2> 1877372 T3320 C978 oasup.LogUpdateProcessor.finish [00004_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id43]} 0 1
   [junit4]   2> 1877373 T3321 C978 oasup.LogUpdateProcessor.finish [00004_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id44]} 0 1
   [junit4]   2> 1877374 T3323 C980 oasup.LogUpdateProcessor.finish [00001_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id45]} 0 1
   [junit4]   2> 1877374 T3319 C980 oasup.LogUpdateProcessor.finish [00001_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id46]} 0 1
   [junit4]   2> 1877375 T3322 C977 oasup.LogUpdateProcessor.finish [00002_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id47]} 0 1
   [junit4]   2> 1877376 T3320 C978 oasup.LogUpdateProcessor.finish [00004_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id48]} 0 1
   [junit4]   2> 1877377 T3321 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id49]} 0 1
   [junit4]   2> 1877378 T3323 C979 oasup.LogUpdateProcessor.finish [00000_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id50]} 0 1
   [junit4]   2> 1877379 T3322 C978 oasup.LogUpdateProcessor.finish [00004_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id52]} 0 1
   [junit4]   2> 1877380 T3319 C980 oasup.LogUpdateProcessor.finish [00001_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id51]} 0 2
   [junit4]   2> 1877380 T3320 C978 oasup.LogUpdateProcessor.finish [00004_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id53]} 0 1
   [junit4]   2> 1877381 T3321 C980 oasup.LogUpdateProcessor.finish [00001_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id54]} 0 1
   [junit4]   2> 1877382 T3323 C980 oasup.LogUpdateProcessor.finish [00001_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id55]} 0 1
   [junit4]   2> 1877383 T3322 C977 oasup.LogUpdateProcessor.finish [00002_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id56]} 0 1
   [junit4]   2> 1877384 T3320 C980 oasup.LogUpdateProcessor.finish [00001_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id58]} 0 1
   [junit4]   2> 1877384 T3319 C979 oasup.LogUpdateProcessor.finish [00000_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id57]} 0 1
   [junit4]   2> 1877385 T3321 C979 oasup.LogUpdateProcessor.finish [00000_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id59]} 0 1
   [junit4]   2> 1877386 T3323 C977 oasup.LogUpdateProcessor.finish [00002_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id60]} 0 1
   [junit4]   2> 1877387 T3322 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id61]} 0 1
   [junit4]   2> 1877388 T3319 C979 oasup.LogUpdateProcessor.finish [00000_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id63]} 0 1
   [junit4]   2> 1877388 T3320 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id62]} 0 1
   [junit4]   2> 1877389 T3321 C980 oasup.LogUpdateProcessor.finish [00001_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id64]} 0 1
   [junit4]   2> 1877390 T3323 C978 oasup.LogUpdateProcessor.finish [00004_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id65]} 0 1
   [junit4]   2> 1877391 T3322 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id66]} 0 1
   [junit4]   2> 1877392 T3319 C979 oasup.LogUpdateProcessor.finish [00000_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id67]} 0 1
   [junit4]   2> 1877393 T3321 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id69]} 0 1
   [junit4]   2> 1877393 T3320 C978 oasup.LogUpdateProcessor.finish [00004_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id68]} 0 1
   [junit4]   2> 1877394 T3323 C977 oasup.LogUpdateProcessor.finish [00002_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id70]} 0 1
   [junit4]   2> 1877395 T3322 C978 oasup.LogUpdateProcessor.finish [00004_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id71]} 0 1
   [junit4]   2> 1877396 T3319 C979 oasup.LogUpdateProcessor.finish [00000_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id72]} 0 1
   [junit4]   2> 1877397 T3320 C979 oasup.LogUpdateProcessor.finish [00000_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id74]} 0 1
   [junit4]   2> 1877397 T3321 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id73]} 0 1
   [junit4]   2> 1877398 T3323 C980 oasup.LogUpdateProcessor.finish [00001_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id75]} 0 1
   [junit4]   2> 1877399 T3322 C980 oasup.LogUpdateProcessor.finish [00001_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id76]} 0 1
   [junit4]   2> 1877400 T3319 C979 oasup.LogUpdateProcessor.finish [00000_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id77]} 0 1
   [junit4]   2> 1877401 T3321 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id79]} 0 1
   [junit4]   2> 1877401 T3320 C979 oasup.LogUpdateProcessor.finish [00000_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id78]} 0 1
   [junit4]   2> 1877402 T3323 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id80]} 0 1
   [junit4]   2> 1877403 T3322 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id81]} 0 1
   [junit4]   2> 1877404 T3319 C977 oasup.LogUpdateProcessor.finish [00002_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id82]} 0 1
   [junit4]   2> 1877405 T3320 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id84]} 0 1
   [junit4]   2> 1877405 T3321 C979 oasup.LogUpdateProcessor.finish [00000_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id83]} 0 1
   [junit4]   2> 1877406 T3323 C978 oasup.LogUpdateProcessor.finish [00004_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id85]} 0 1
   [junit4]   2> 1877407 T3322 C978 oasup.LogUpdateProcessor.finish [00004_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id86]} 0 1
   [junit4]   2> 1877408 T3319 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id87]} 0 1
   [junit4]   2> 1877409 T3320 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id88]} 0 1
   [junit4]   2> 1877409 T3321 C977 oasup.LogUpdateProcessor.finish [00002_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id89]} 0 1
   [junit4]   2> 1877410 T3323 C980 oasup.LogUpdateProcessor.finish [00001_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id90]} 0 1
   [junit4]   2> 1877411 T3322 C979 oasup.LogUpdateProcessor.finish [00000_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id91]} 0 1
   [junit4]   2> 1877412 T3319 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id92]} 0 1
   [junit4]   2> 1877413 T3320 C977 oasup.LogUpdateProcessor.finish [00002_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id93]} 0 1
   [junit4]   2> 1877413 T3321 C980 oasup.LogUpdateProcessor.finish [00001_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id94]} 0 1
   [junit4]   2> 1877414 T3323 C977 oasup.LogUpdateProcessor.finish [00002_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id95]} 0 1
   [junit4]   2> 1877415 T3322 C977 oasup.LogUpdateProcessor.finish [00002_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id96]} 0 1
   [junit4]   2> 1877416 T3319 C978 oasup.LogUpdateProcessor.finish [00004_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id97]} 0 1
   [junit4]   2> 1877417 T3320 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id98]} 0 1
   [junit4]   2> 1877418 T3323 C980 oasup.LogUpdateProcessor.finish [00001_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id100]} 0 1
   [junit4]   2> 1877418 T3321 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id99]} 0 1
   [junit4]   2> 1877419 T3322 C977 oasup.LogUpdateProcessor.finish [00002_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id101]} 0 1
   [junit4]   2> 1877420 T3319 C980 oasup.LogUpdateProcessor.finish [00001_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id102]} 0 1
   [junit4]   2> 1877421 T3320 C978 oasup.LogUpdateProcessor.finish [00004_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id103]} 0 1
   [junit4]   2> 1877422 T3321 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id105]} 0 1
   [junit4]   2> 1877422 T3323 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id104]} 0 1
   [junit4]   2> 1877423 T3322 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id106]} 0 1
   [junit4]   2> 1877424 T3319 C978 oasup.LogUpdateProcessor.finish [00004_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id107]} 0 1
   [junit4]   2> 1877425 T3320 C976 oasup.LogUpdateProcessor.finish [00003_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id108]} 0 1
   [junit4]   2> 1877426 T3321 C977 oasup.LogUpdateProcessor.finish [00002_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id109]} 0 1
   [junit4]   2> 1877426 T3323 C977 oasup.LogUpdateProcessor.finish [00002_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id110]} 0 1
   [junit4]   2> 1877427 T3322 C977 oasup.LogUpdateProcessor.finish [00002_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id111]} 0 1
   [junit4]   2> 1877429 T3320 C979 oasup.LogUpdateProcessor.finish [00000_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id113]} 0 1
   [junit4]   2> 1877429 T3319 C980 oasup.LogUpdateProcessor.finish [00001_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id112]} 0 2
   [junit4]   2> 1877430 T3323 C979 oasup.LogUpdateProcessor.finish [00000_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id115]} 0 1
   [junit4]   2> 1877430 T3321 C979 oasup.LogUpdateProcessor.finish [00000_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id114]} 0 1
   [junit4]   2> 1877431 T3322 C977 oasup.LogUpdateProcessor.finish [00002_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id116]} 0 1
   [junit4]   2> 1877433 T3320 C978 oasup.LogUpdateProcessor.finish [00004_core] webapp=/solr path=/update params={wt=javabin&version=2} {add=[id117]} 0 1
   [junit4]   2> 1877434 T3323 C979 oas

[...truncated too long message...]

.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:724)
   [junit4]    >    4) Thread[id=3379, name=commitScheduler-2495-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:724)
   [junit4]    >    5) Thread[id=3345, name=commitScheduler-2476-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:724)
   [junit4]    >    6) Thread[id=3349, name=commitScheduler-2479-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:724)
   [junit4]    >    7) Thread[id=3372, name=commitScheduler-2485-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:724)
   [junit4]    >    8) Thread[id=3376, name=commitScheduler-2491-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:724)
   [junit4]    >    9) Thread[id=3344, name=commitScheduler-2475-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:724)
   [junit4]    >   10) Thread[id=3346, name=commitScheduler-2473-thread-1, state=WAITING, group=TGRP-OpenCloseCoreStressTest]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.park(LockSupport.java:186)
   [junit4]    >         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2043)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1079)
   [junit4]    >         at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:807)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1068)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
   [junit4]    >         at java.lang.Thread.run(Thread.java:724)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([3F6997CF3A2C204]:0)
   [junit4] Completed on J0 in 54.81s, 6 tests, 2 errors, 4 skipped <<< FAILURES!

[...truncated 451 lines...]
BUILD FAILED
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:429: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:409: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/build.xml:39: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/build.xml:189: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/solr/common-build.xml:489: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:1276: The following error occurred while executing this line:
/usr/home/hudson/hudson-slave/workspace/Lucene-Solr-Tests-4.x-Java7/lucene/common-build.xml:912: There were test failures: 336 suites, 1478 tests, 2 suite-level errors, 654 ignored (5 assumptions)

Total time: 60 minutes 15 seconds
Build step 'Invoke Ant' marked build as failure
Archiving artifacts
Recording test results
Email was triggered for: Failure
Sending email for trigger: Failure