You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by Policeman Jenkins Server <je...@thetaphi.de> on 2015/01/08 18:19:49 UTC

[JENKINS] Lucene-Solr-5.x-Linux (64bit/jdk1.9.0-ea-b44) - Build # 11381 - Failure!

Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/11381/
Java: 64bit/jdk1.9.0-ea-b44 -XX:+UseCompressedOops -XX:+UseG1GC

55 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.lucene.search.TestSearcherManager

Error Message:
8 threads leaked from SUITE scope at org.apache.lucene.search.TestSearcherManager:     1) Thread[id=715, name=TestSearcherManager-17-thread-6, state=TIMED_WAITING, group=TGRP-TestSearcherManager]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=716, name=TestSearcherManager-17-thread-7, state=TIMED_WAITING, group=TGRP-TestSearcherManager]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=710, name=TestSearcherManager-17-thread-1, state=TIMED_WAITING, group=TGRP-TestSearcherManager]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=717, name=TestSearcherManager-17-thread-8, state=TIMED_WAITING, group=TGRP-TestSearcherManager]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=712, name=TestSearcherManager-17-thread-4, state=TIMED_WAITING, group=TGRP-TestSearcherManager]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=713, name=TestSearcherManager-17-thread-3, state=TIMED_WAITING, group=TGRP-TestSearcherManager]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=714, name=TestSearcherManager-17-thread-5, state=TIMED_WAITING, group=TGRP-TestSearcherManager]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=711, name=TestSearcherManager-17-thread-2, state=TIMED_WAITING, group=TGRP-TestSearcherManager]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 8 threads leaked from SUITE scope at org.apache.lucene.search.TestSearcherManager: 
   1) Thread[id=715, name=TestSearcherManager-17-thread-6, state=TIMED_WAITING, group=TGRP-TestSearcherManager]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=716, name=TestSearcherManager-17-thread-7, state=TIMED_WAITING, group=TGRP-TestSearcherManager]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=710, name=TestSearcherManager-17-thread-1, state=TIMED_WAITING, group=TGRP-TestSearcherManager]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=717, name=TestSearcherManager-17-thread-8, state=TIMED_WAITING, group=TGRP-TestSearcherManager]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=712, name=TestSearcherManager-17-thread-4, state=TIMED_WAITING, group=TGRP-TestSearcherManager]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=713, name=TestSearcherManager-17-thread-3, state=TIMED_WAITING, group=TGRP-TestSearcherManager]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=714, name=TestSearcherManager-17-thread-5, state=TIMED_WAITING, group=TGRP-TestSearcherManager]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=711, name=TestSearcherManager-17-thread-2, state=TIMED_WAITING, group=TGRP-TestSearcherManager]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([DCAA4FA2B5A6C7D0]:0)


FAILED:  junit.framework.TestSuite.org.apache.lucene.search.TestSearcherManager

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=715, name=TestSearcherManager-17-thread-6, state=TIMED_WAITING, group=TGRP-TestSearcherManager]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=716, name=TestSearcherManager-17-thread-7, state=TIMED_WAITING, group=TGRP-TestSearcherManager]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=710, name=TestSearcherManager-17-thread-1, state=TIMED_WAITING, group=TGRP-TestSearcherManager]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=717, name=TestSearcherManager-17-thread-8, state=TIMED_WAITING, group=TGRP-TestSearcherManager]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=712, name=TestSearcherManager-17-thread-4, state=TIMED_WAITING, group=TGRP-TestSearcherManager]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=713, name=TestSearcherManager-17-thread-3, state=TIMED_WAITING, group=TGRP-TestSearcherManager]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=714, name=TestSearcherManager-17-thread-5, state=TIMED_WAITING, group=TGRP-TestSearcherManager]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=711, name=TestSearcherManager-17-thread-2, state=TIMED_WAITING, group=TGRP-TestSearcherManager]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=715, name=TestSearcherManager-17-thread-6, state=TIMED_WAITING, group=TGRP-TestSearcherManager]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=716, name=TestSearcherManager-17-thread-7, state=TIMED_WAITING, group=TGRP-TestSearcherManager]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=710, name=TestSearcherManager-17-thread-1, state=TIMED_WAITING, group=TGRP-TestSearcherManager]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=717, name=TestSearcherManager-17-thread-8, state=TIMED_WAITING, group=TGRP-TestSearcherManager]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=712, name=TestSearcherManager-17-thread-4, state=TIMED_WAITING, group=TGRP-TestSearcherManager]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=713, name=TestSearcherManager-17-thread-3, state=TIMED_WAITING, group=TGRP-TestSearcherManager]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=714, name=TestSearcherManager-17-thread-5, state=TIMED_WAITING, group=TGRP-TestSearcherManager]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=711, name=TestSearcherManager-17-thread-2, state=TIMED_WAITING, group=TGRP-TestSearcherManager]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([DCAA4FA2B5A6C7D0]:0)


FAILED:  org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat2.testTTFBlockSize

Error Message:
On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.codecs.lucene50.TestBlockPostingsFormat2 DCAA4FA2B5A6C7D0-001/testDFBlockSize-001: Is a directory

Stack Trace:
java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.codecs.lucene50.TestBlockPostingsFormat2 DCAA4FA2B5A6C7D0-001/testDFBlockSize-001: Is a directory
	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:418)
	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:236)
	at org.apache.lucene.store.MockDirectoryWrapper.renameFile(MockDirectoryWrapper.java:268)
	at org.apache.lucene.index.SegmentInfos.finishCommit(SegmentInfos.java:727)
	at org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:2914)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2895)
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2855)
	at org.apache.lucene.index.RandomIndexWriter.maybeCommit(RandomIndexWriter.java:158)
	at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:149)
	at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:107)
	at org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat2.testTTFBlockSize(TestBlockPostingsFormat2.java:114)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat2.testTTFBlockSizeMultiple

Error Message:
On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.codecs.lucene50.TestBlockPostingsFormat2 DCAA4FA2B5A6C7D0-001/testDFBlockSize-002: Is a directory

Stack Trace:
java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.codecs.lucene50.TestBlockPostingsFormat2 DCAA4FA2B5A6C7D0-001/testDFBlockSize-002: Is a directory
	at __randomizedtesting.SeedInfo.seed([DCAA4FA2B5A6C7D0:5D8A39B845228F80]:0)
	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:418)
	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:236)
	at org.apache.lucene.store.MockDirectoryWrapper.renameFile(MockDirectoryWrapper.java:268)
	at org.apache.lucene.index.SegmentInfos.finishCommit(SegmentInfos.java:727)
	at org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:2914)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2895)
	at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:965)
	at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1008)
	at org.apache.lucene.index.RandomIndexWriter.close(RandomIndexWriter.java:369)
	at org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat2.tearDown(TestBlockPostingsFormat2.java:55)
	at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:885)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat2.testDFBlockSizeMultiple

Error Message:
On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.codecs.lucene50.TestBlockPostingsFormat2 DCAA4FA2B5A6C7D0-001/testDFBlockSize-003: Is a directory

Stack Trace:
java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.codecs.lucene50.TestBlockPostingsFormat2 DCAA4FA2B5A6C7D0-001/testDFBlockSize-003: Is a directory
	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:418)
	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:236)
	at org.apache.lucene.store.MockDirectoryWrapper.renameFile(MockDirectoryWrapper.java:268)
	at org.apache.lucene.index.SegmentInfos.finishCommit(SegmentInfos.java:727)
	at org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:2914)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2895)
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2855)
	at org.apache.lucene.index.RandomIndexWriter.maybeCommit(RandomIndexWriter.java:158)
	at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:149)
	at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:107)
	at org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat2.testDFBlockSizeMultiple(TestBlockPostingsFormat2.java:103)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat2.testDFBlockSize

Error Message:
On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.codecs.lucene50.TestBlockPostingsFormat2 DCAA4FA2B5A6C7D0-001/testDFBlockSize-004: Is a directory

Stack Trace:
java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.codecs.lucene50.TestBlockPostingsFormat2 DCAA4FA2B5A6C7D0-001/testDFBlockSize-004: Is a directory
	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:418)
	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:236)
	at org.apache.lucene.store.MockDirectoryWrapper.renameFile(MockDirectoryWrapper.java:268)
	at org.apache.lucene.index.SegmentInfos.finishCommit(SegmentInfos.java:727)
	at org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:2914)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2895)
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2855)
	at org.apache.lucene.index.RandomIndexWriter.maybeCommit(RandomIndexWriter.java:158)
	at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:149)
	at org.apache.lucene.index.RandomIndexWriter.addDocument(RandomIndexWriter.java:107)
	at org.apache.lucene.codecs.lucene50.TestBlockPostingsFormat2.testDFBlockSize(TestBlockPostingsFormat2.java:92)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.lucene.codecs.lucene50.TestLucene50DocValuesFormat.testSortedSetVariableLengthSingleValuedVsStoredFields

Error Message:
On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J0/temp/lucene.codecs.lucene50.TestLucene50DocValuesFormat DCAA4FA2B5A6C7D0-001/dvduel-001: Is a directory

Stack Trace:
java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J0/temp/lucene.codecs.lucene50.TestLucene50DocValuesFormat DCAA4FA2B5A6C7D0-001/dvduel-001: Is a directory
	at __randomizedtesting.SeedInfo.seed([DCAA4FA2B5A6C7D0:2CD904949FA5D723]:0)
	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:418)
	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:236)
	at org.apache.lucene.store.MockDirectoryWrapper.renameFile(MockDirectoryWrapper.java:268)
	at org.apache.lucene.index.SegmentInfos.finishCommit(SegmentInfos.java:727)
	at org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:2914)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2895)
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2855)
	at org.apache.lucene.index.RandomIndexWriter.commit(RandomIndexWriter.java:257)
	at org.apache.lucene.index.BaseDocValuesFormatTestCase.doTestSortedSetVsStoredFields(BaseDocValuesFormatTestCase.java:1977)
	at org.apache.lucene.index.BaseDocValuesFormatTestCase.testSortedSetVariableLengthSingleValuedVsStoredFields(BaseDocValuesFormatTestCase.java:2129)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.lucene.codecs.lucene50.TestLucene50DocValuesFormat.testSortedSetFixedLengthSingleValuedVsStoredFields

Error Message:
On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J0/temp/lucene.codecs.lucene50.TestLucene50DocValuesFormat DCAA4FA2B5A6C7D0-001/dvduel-002: Is a directory

Stack Trace:
java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J0/temp/lucene.codecs.lucene50.TestLucene50DocValuesFormat DCAA4FA2B5A6C7D0-001/dvduel-002: Is a directory
	at __randomizedtesting.SeedInfo.seed([DCAA4FA2B5A6C7D0:58DBFA4456DF5CCD]:0)
	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:418)
	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:236)
	at org.apache.lucene.store.MockDirectoryWrapper.renameFile(MockDirectoryWrapper.java:268)
	at org.apache.lucene.index.SegmentInfos.finishCommit(SegmentInfos.java:727)
	at org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:2914)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2895)
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2855)
	at org.apache.lucene.index.RandomIndexWriter.commit(RandomIndexWriter.java:257)
	at org.apache.lucene.index.BaseDocValuesFormatTestCase.doTestSortedSetVsStoredFields(BaseDocValuesFormatTestCase.java:1977)
	at org.apache.lucene.index.BaseDocValuesFormatTestCase.testSortedSetFixedLengthSingleValuedVsStoredFields(BaseDocValuesFormatTestCase.java:2121)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.lucene.codecs.lucene50.TestLucene50DocValuesFormat.testSortedVariableLengthBigVsStoredFields

Error Message:
On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J0/temp/lucene.codecs.lucene50.TestLucene50DocValuesFormat DCAA4FA2B5A6C7D0-001/dvduel-003: Is a directory

Stack Trace:
java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J0/temp/lucene.codecs.lucene50.TestLucene50DocValuesFormat DCAA4FA2B5A6C7D0-001/dvduel-003: Is a directory
	at __randomizedtesting.SeedInfo.seed([DCAA4FA2B5A6C7D0:2C5A08C786BD625F]:0)
	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:418)
	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:236)
	at org.apache.lucene.store.MockDirectoryWrapper.renameFile(MockDirectoryWrapper.java:268)
	at org.apache.lucene.index.SegmentInfos.finishCommit(SegmentInfos.java:727)
	at org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:2914)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2895)
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2855)
	at org.apache.lucene.index.RandomIndexWriter.commit(RandomIndexWriter.java:257)
	at org.apache.lucene.index.BaseDocValuesFormatTestCase.doTestSortedVsStoredFields(BaseDocValuesFormatTestCase.java:1446)
	at org.apache.lucene.codecs.lucene50.TestLucene50DocValuesFormat.testSortedVariableLengthBigVsStoredFields(TestLucene50DocValuesFormat.java:82)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.lucene.codecs.lucene50.TestLucene50DocValuesFormat.testTermsEnumFixedWidth

Error Message:
On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J0/temp/lucene.codecs.lucene50.TestLucene50DocValuesFormat DCAA4FA2B5A6C7D0-001/tempDir-001: Is a directory

Stack Trace:
java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J0/temp/lucene.codecs.lucene50.TestLucene50DocValuesFormat DCAA4FA2B5A6C7D0-001/tempDir-001: Is a directory
	at __randomizedtesting.SeedInfo.seed([DCAA4FA2B5A6C7D0:21AAF479EFBF429C]:0)
	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:418)
	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:236)
	at org.apache.lucene.store.FilterDirectory.renameFile(FilterDirectory.java:83)
	at org.apache.lucene.index.SegmentInfos.finishCommit(SegmentInfos.java:727)
	at org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:2914)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2895)
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2855)
	at org.apache.lucene.index.RandomIndexWriter.commit(RandomIndexWriter.java:257)
	at org.apache.lucene.codecs.lucene50.TestLucene50DocValuesFormat.doTestTermsEnumRandom(TestLucene50DocValuesFormat.java:170)
	at org.apache.lucene.codecs.lucene50.TestLucene50DocValuesFormat.testTermsEnumFixedWidth(TestLucene50DocValuesFormat.java:98)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.lucene.codecs.lucene50.TestLucene50DocValuesFormat.testSortedSetFixedLengthVsStoredFields

Error Message:
On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J0/temp/lucene.codecs.lucene50.TestLucene50DocValuesFormat DCAA4FA2B5A6C7D0-001/dvduel-004: Is a directory

Stack Trace:
java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J0/temp/lucene.codecs.lucene50.TestLucene50DocValuesFormat DCAA4FA2B5A6C7D0-001/dvduel-004: Is a directory
	at __randomizedtesting.SeedInfo.seed([DCAA4FA2B5A6C7D0:A9D9FE6EA25B9F86]:0)
	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:418)
	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:236)
	at org.apache.lucene.store.MockDirectoryWrapper.renameFile(MockDirectoryWrapper.java:268)
	at org.apache.lucene.index.SegmentInfos.finishCommit(SegmentInfos.java:727)
	at org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:2914)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2895)
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2855)
	at org.apache.lucene.index.RandomIndexWriter.commit(RandomIndexWriter.java:257)
	at org.apache.lucene.index.BaseDocValuesFormatTestCase.doTestSortedSetVsStoredFields(BaseDocValuesFormatTestCase.java:1977)
	at org.apache.lucene.index.BaseDocValuesFormatTestCase.testSortedSetFixedLengthVsStoredFields(BaseDocValuesFormatTestCase.java:2041)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.lucene.codecs.lucene50.TestLucene50DocValuesFormat.testTermsEnumVariableWidth

Error Message:
On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J0/temp/lucene.codecs.lucene50.TestLucene50DocValuesFormat DCAA4FA2B5A6C7D0-001/tempDir-002: Is a directory

Stack Trace:
java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J0/temp/lucene.codecs.lucene50.TestLucene50DocValuesFormat DCAA4FA2B5A6C7D0-001/tempDir-002: Is a directory
	at __randomizedtesting.SeedInfo.seed([DCAA4FA2B5A6C7D0:19965B5D0819F99B]:0)
	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:418)
	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:236)
	at org.apache.lucene.store.MockDirectoryWrapper.renameFile(MockDirectoryWrapper.java:268)
	at org.apache.lucene.index.SegmentInfos.finishCommit(SegmentInfos.java:727)
	at org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:2914)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2895)
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2855)
	at org.apache.lucene.index.RandomIndexWriter.commit(RandomIndexWriter.java:257)
	at org.apache.lucene.codecs.lucene50.TestLucene50DocValuesFormat.doTestTermsEnumRandom(TestLucene50DocValuesFormat.java:170)
	at org.apache.lucene.codecs.lucene50.TestLucene50DocValuesFormat.testTermsEnumVariableWidth(TestLucene50DocValuesFormat.java:106)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.lucene.codecs.lucene50.TestLucene50DocValuesFormat.testOneFloat

Error Message:
On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J0/temp/lucene.codecs.lucene50.TestLucene50DocValuesFormat DCAA4FA2B5A6C7D0-001/index-NIOFSDirectory-001: Is a directory

Stack Trace:
java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J0/temp/lucene.codecs.lucene50.TestLucene50DocValuesFormat DCAA4FA2B5A6C7D0-001/index-NIOFSDirectory-001: Is a directory
	at __randomizedtesting.SeedInfo.seed([DCAA4FA2B5A6C7D0:5F19C56DB2B28ABC]:0)
	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:418)
	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:236)
	at org.apache.lucene.store.MockDirectoryWrapper.renameFile(MockDirectoryWrapper.java:268)
	at org.apache.lucene.index.SegmentInfos.finishCommit(SegmentInfos.java:727)
	at org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:2914)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2895)
	at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:965)
	at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1008)
	at org.apache.lucene.index.RandomIndexWriter.close(RandomIndexWriter.java:369)
	at org.apache.lucene.index.BaseDocValuesFormatTestCase.testOneFloat(BaseDocValuesFormatTestCase.java:136)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.lucene.codecs.lucene50.TestLucene50DocValuesFormat.testSortedSetVariableLengthVsStoredFields

Error Message:
On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J0/temp/lucene.codecs.lucene50.TestLucene50DocValuesFormat DCAA4FA2B5A6C7D0-001/dvduel-005: Is a directory

Stack Trace:
java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J0/temp/lucene.codecs.lucene50.TestLucene50DocValuesFormat DCAA4FA2B5A6C7D0-001/dvduel-005: Is a directory
	at __randomizedtesting.SeedInfo.seed([DCAA4FA2B5A6C7D0:F8BB1E5ED15AD07D]:0)
	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:418)
	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:236)
	at org.apache.lucene.store.MockDirectoryWrapper.renameFile(MockDirectoryWrapper.java:268)
	at org.apache.lucene.index.SegmentInfos.finishCommit(SegmentInfos.java:727)
	at org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:2914)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2895)
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2855)
	at org.apache.lucene.index.RandomIndexWriter.commit(RandomIndexWriter.java:257)
	at org.apache.lucene.index.BaseDocValuesFormatTestCase.doTestSortedSetVsStoredFields(BaseDocValuesFormatTestCase.java:1977)
	at org.apache.lucene.index.BaseDocValuesFormatTestCase.testSortedSetVariableLengthVsStoredFields(BaseDocValuesFormatTestCase.java:2112)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:3

[...truncated too long message...]

.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    6) Thread[id=713, name=TestSearcherManager-17-thread-3, state=TIMED_WAITING, group=TGRP-TestSearcherManager]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    7) Thread[id=714, name=TestSearcherManager-17-thread-5, state=TIMED_WAITING, group=TGRP-TestSearcherManager]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    8) Thread[id=711, name=TestSearcherManager-17-thread-2, state=TIMED_WAITING, group=TGRP-TestSearcherManager]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DCAA4FA2B5A6C7D0]:0)
   [junit4] Completed on J0 in 26.30s, 9 tests, 3 errors <<< FAILURES!

[...truncated 123 lines...]
   [junit4] Suite: org.apache.lucene.index.TestIndexWriterReader
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestIndexWriterReader -Dtests.method=testIsCurrent -Dtests.seed=DCAA4FA2B5A6C7D0 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=es_US -Dtests.timezone=America/Grand_Turk -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.01s J1 | TestIndexWriterReader.testIsCurrent <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterReader DCAA4FA2B5A6C7D0-001/index-MMapDirectory-001: Is a directory
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([DCAA4FA2B5A6C7D0:4E36153F92CA809F]:0)
   [junit4]    > 	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:418)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:236)
   [junit4]    > 	at org.apache.lucene.store.MockDirectoryWrapper.renameFile(MockDirectoryWrapper.java:268)
   [junit4]    > 	at org.apache.lucene.index.SegmentInfos.finishCommit(SegmentInfos.java:727)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:2914)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2895)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:965)
   [junit4]    > 	at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1008)
   [junit4]    > 	at org.apache.lucene.index.TestIndexWriterReader.testIsCurrent(TestIndexWriterReader.java:197)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4] IGNOR/A 0.00s J1 | TestIndexWriterReader.testDuringAddIndexes
   [junit4]    > Assumption #1: 'nightly' test group is disabled (@Nightly())
   [junit4]   2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.index.TestIndexWriterReader DCAA4FA2B5A6C7D0-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50): {field1=PostingsFormat(name=LuceneVarGapFixedInterval), field=Lucene50(blocksize=128), indexname=PostingsFormat(name=Asserting), foo=Lucene50(blocksize=128), id=PostingsFormat(name=Asserting), field3=PostingsFormat(name=Asserting), field2=FST50, field5=PostingsFormat(name=LuceneVarGapFixedInterval), field4=Lucene50(blocksize=128)}, docValues:{}, sim=DefaultSimilarity, locale=es_US, timezone=America/Grand_Turk
   [junit4]   2> NOTE: Linux 3.13.0-43-generic amd64/Oracle Corporation 1.9.0-ea (64-bit)/cpus=8,threads=1,free=355646744,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [TestDocValues, TestQueryWrapperFilter, TestNIOFSDirectory, TestSimpleExplanationsOfNonMatches, TestTimSorter, TestMultiPhraseQuery, TestLevenshteinAutomata, TestThreadedForceMerge, TestCrash, TestDocTermOrdsRangeFilter, TestDirectPacked, TestUsageTrackingFilterCachingPolicy, TestPrefixInBooleanQuery, TestTerm, TestBagOfPositions, TestMinShouldMatch2, TestTotalHitCountCollector, TestIndexWriterCommit, TestComplexExplanationsOfNonMatches, TestSpans, TestEliasFanoSequence, TestDocValuesScoring, TestPrefixFilter, TestMinimize, TestSegmentReader, TestShardSearching, TestSimpleAttributeImpl, TestOfflineSorter, TestMixedCodecs, TestNoMergePolicy, TestLongPostings, TestDocTermOrdsRewriteMethod, TestNamedSPILoader, TestCharsRef, FuzzyTermOnShortTermsTest, TestCodecHoldsOpenFiles, TestIndexWriterWithThreads, TestParallelTermEnum, TestPayloadsOnVectors, TestSameTokenSamePosition, TestSparseFixedBitSet, TestBinaryDocValuesUpdates, TestFieldCacheRewriteMethod, TestFastCompressionMode, TestHighCompressionMode, TestMmapDirectory, TestSubScorerFreqs, TestTieredMergePolicy, TestBitUtil, TestMultiDocValues, TestSearchWithThreads, TestForTooMuchCloning, TestIndexWriterMergePolicy, Test2BPositions, TestMultiMMap, TestFieldCacheTermsFilter, TestNumericRangeQuery64, TestSpanSearchEquivalence, TestPayloadTermQuery, TestIndexFileDeleter, TestIntroSorter, TestNumericTokenStream, TestFilterCachingPolicy, TestAutomatonQuery, TestLockFactory, TestBinaryDocument, TestIndexInput, TestNGramPhraseQuery, TestAllFilesHaveChecksumFooter, TestCustomSearcherSort, TestTopScoreDocCollector, TestSparseFixedBitDocIdSet, TestMockFilesystems, TestFlushByRamOrCountsPolicy, TestFuzzyQuery, TestSimilarity, TestRollingBuffer, TestCachingTokenFilter, TestAssertingTermVectorsFormat, TestTopFieldCollector, TestFlex, TestLRUFilterCache, TestMutableValues, TestPagedBytes, TestDisjunctionMaxQuery, TestManyFields, TestToken, TestDeletionPolicy, TestIsCurrent, TestIndexWriterOutOfMemory, TestSentinelIntSet, TestGraphTokenizers, TestMultiTermQueryRewrites, TestAtomicUpdate, TestCharFilter, TestTermRangeQuery, TestPhraseQuery, TestBlockPostingsFormat2, TestBooleanScorer, TestPerFieldDocValuesFormat, TestIndexWriterReader]
   [junit4] Completed on J1 in 8.27s, 23 tests, 1 failure, 1 skipped <<< FAILURES!

[...truncated 549 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:529: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:477: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:61: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/extra-targets.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build.xml:49: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:1363: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:970: There were test failures: 409 suites, 3188 tests, 2 suite-level errors, 3 errors, 50 failures, 1660 ignored (29 assumptions)

Total time: 5 minutes 2 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 64bit/jdk1.9.0-ea-b44 -XX:+UseCompressedOops -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-5.x-Linux (32bit/jdk1.7.0_72) - Build # 11384 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/11384/
Java: 32bit/jdk1.7.0_72 -client -XX:+UseG1GC

1 tests failed.
FAILED:  org.apache.solr.core.TestDynamicLoading.testDistribSearch

Error Message:
{   "responseHeader":{     "status":404,     "QTime":3},   "error":{     "msg":"no such blob or version available: test/1",     "code":404}}

Stack Trace:
java.lang.AssertionError: {
  "responseHeader":{
    "status":404,
    "QTime":3},
  "error":{
    "msg":"no such blob or version available: test/1",
    "code":404}}
	at __randomizedtesting.SeedInfo.seed([A5CA2100C118E8DA:242CAF18B64788E6]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.core.TestDynamicLoading.dynamicLoading(TestDynamicLoading.java:108)
	at org.apache.solr.core.TestDynamicLoading.doTest(TestDynamicLoading.java:64)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:868)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 9471 lines...]
   [junit4] Suite: org.apache.solr.core.TestDynamicLoading
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/init-core-data-001
   [junit4]   2> 690871 T3787 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 690872 T3787 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 690876 T3787 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 690877 T3787 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 690877 T3788 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 690977 T3787 oasc.ZkTestServer.run start zk server on port:58752
   [junit4]   2> 690978 T3787 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 690981 T3787 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 690984 T3795 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@cd0b80 name:ZooKeeperConnection Watcher:127.0.0.1:58752 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 690984 T3787 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 690985 T3787 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 690985 T3787 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 690990 T3787 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 690991 T3787 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 690993 T3798 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1361c3f name:ZooKeeperConnection Watcher:127.0.0.1:58752/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 690993 T3787 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 690993 T3787 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 690994 T3787 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 690996 T3787 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 690998 T3787 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 691000 T3787 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 691001 T3787 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 691002 T3787 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 691004 T3787 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 691005 T3787 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 691007 T3787 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 691008 T3787 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 691009 T3787 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 691010 T3787 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 691011 T3787 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 691012 T3787 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 691013 T3787 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 691014 T3787 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 691016 T3787 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 691016 T3787 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 691018 T3787 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 691019 T3787 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 691020 T3787 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 691021 T3787 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 691022 T3787 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 691023 T3787 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 691024 T3787 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 691025 T3787 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 691294 T3787 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 691340 T3787 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:?????
   [junit4]   2> 691340 T3787 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@47034b
   [junit4]   2> 691341 T3787 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 691342 T3787 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-002
   [junit4]   2> 691342 T3787 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-002/'
   [junit4]   2> 691363 T3787 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-002/solr.xml
   [junit4]   2> 691398 T3787 oasc.CoreContainer.<init> New CoreContainer 26407699
   [junit4]   2> 691398 T3787 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-002/]
   [junit4]   2> 691399 T3787 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 691400 T3787 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 691400 T3787 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 691401 T3787 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 691401 T3787 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 691401 T3787 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 691402 T3787 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 691402 T3787 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 691402 T3787 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 691403 T3787 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 691403 T3787 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 691404 T3787 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 691404 T3787 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 691404 T3787 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 691405 T3787 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58752/solr
   [junit4]   2> 691405 T3787 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 691405 T3787 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 691406 T3787 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 691408 T3810 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@5e074b name:ZooKeeperConnection Watcher:127.0.0.1:58752 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 691409 T3787 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 691409 T3787 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 691412 T3787 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 691413 T3813 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@11bf3b6 name:ZooKeeperConnection Watcher:127.0.0.1:58752/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 691414 T3787 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 691416 T3787 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 691422 T3787 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 691425 T3787 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 691429 T3787 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 691433 T3787 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 691437 T3787 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 691440 T3787 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:56919_
   [junit4]   2> 691442 T3787 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:56919_
   [junit4]   2> 691446 T3787 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 691450 T3787 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 691453 T3787 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 691457 T3787 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:56919_
   [junit4]   2> 691457 T3787 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 691461 T3787 oasc.Overseer.start Overseer (id=93110379544969219-127.0.0.1:56919_-n_0000000000) starting
   [junit4]   2> 691465 T3787 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 691476 T3787 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 691477 T3815 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 691478 T3787 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 691482 T3787 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 691484 T3787 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 691491 T3814 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 691495 T3817 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
   [junit4]   2> 691496 T3817 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 691497 T3813 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 691497 T3817 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 691504 T3814 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 0, message = {
   [junit4]   2> 	  "base_url":"http://127.0.0.1:56919",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "node_name":"127.0.0.1:56919_",
   [junit4]   2> 	  "state":"down"} current state version: 0
   [junit4]   2> 691504 T3814 oasco.ReplicaMutator.updateState Update state numShards=1 message={
   [junit4]   2> 	  "base_url":"http://127.0.0.1:56919",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "node_name":"127.0.0.1:56919_",
   [junit4]   2> 	  "state":"down"}
   [junit4]   2> 691505 T3814 oasco.ClusterStateMutator.createCollection building a new cName: control_collection
   [junit4]   2> 691505 T3814 oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 691507 T3813 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 692498 T3817 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 692499 T3817 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 692500 T3817 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 692500 T3817 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 692502 T3817 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 692502 T3817 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-002/collection1/'
   [junit4]   2> 692504 T3817 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading%20A5CA2100C118E8DA-001/tempDir-002/collection1/lib/.svn/' to classloader
   [junit4]   2> 692504 T3817 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading%20A5CA2100C118E8DA-001/tempDir-002/collection1/lib/classes/' to classloader
   [junit4]   2> 692505 T3817 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading%20A5CA2100C118E8DA-001/tempDir-002/collection1/lib/README' to classloader
   [junit4]   2> 692527 T3817 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 692530 T3817 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 692537 T3817 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 692553 T3817 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 692587 T3817 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 692588 T3817 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 692598 T3817 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 692920 T3817 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 692925 T3817 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 692927 T3817 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 692936 T3817 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 692939 T3817 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 692942 T3817 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 692942 T3817 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 692943 T3817 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 692943 T3817 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 692944 T3817 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 692944 T3817 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 692944 T3817 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 692944 T3817 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 692945 T3817 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-002/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-001/control/data/
   [junit4]   2> 692945 T3817 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@17ba1a8
   [junit4]   2> 692945 T3817 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-001/control/data
   [junit4]   2> 692946 T3817 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-001/control/data/index/
   [junit4]   2> 692946 T3817 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 692946 T3817 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-001/control/data/index
   [junit4]   2> 692947 T3817 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=34, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 692947 T3817 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@1f73de5 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@100b8a8),segFN=segments_1,generation=1}
   [junit4]   2> 692948 T3817 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 692950 T3817 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 692951 T3817 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 692951 T3817 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 692951 T3817 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 692951 T3817 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 692952 T3817 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 692952 T3817 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 692952 T3817 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 692952 T3817 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 692953 T3817 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 692953 T3817 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 692954 T3817 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 692954 T3817 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 692955 T3817 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
   [junit4]   2> 692955 T3817 oasc.RequestHandlers.initHandlersFromConfig created /schema: org.apache.solr.handler.SchemaHandler
   [junit4]   2> 692956 T3817 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler
   [junit4]   2> 692956 T3817 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler
   [junit4]   2> 692957 T3817 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
   [junit4]   2> 692957 T3817 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
   [junit4]   2> 692958 T3817 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
   [junit4]   2> 692958 T3817 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
   [junit4]   2> 692959 T3817 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
   [junit4]   2> 692959 T3817 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler
   [junit4]   2> 692959 T3817 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
   [junit4]   2> 692960 T3817 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler
   [junit4]   2> 692960 T3817 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 692961 T3817 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 692963 T3817 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 692965 T3817 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 692966 T3817 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 692967 T3817 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 692968 T3817 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 692969 T3817 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 692969 T3817 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 692970 T3817 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@1f8d787
   [junit4]   2> 692970 T3817 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@1f73de5 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@100b8a8),segFN=segments_1,generation=1}
   [junit4]   2> 692971 T3817 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 692971 T3817 oass.SolrIndexSearcher.<init> Opening Searcher@38e6e8[collection1] main
   [junit4]   2> 692971 T3817 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 692972 T3817 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 692972 T3817 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 692973 T3817 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 692973 T3817 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 692973 T3817 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 692974 T3817 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 692974 T3817 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 692974 T3817 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 692975 T3817 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 692975 T3817 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 692975 T3818 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@38e6e8[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 692976 T3821 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:56919 collection:control_collection shard:shard1
   [junit4]   2> 692976 T3787 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 692976 T3821 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 692976 T3787 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 692977 T3787 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 692978 T3787 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 692981 T3821 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 692982 T3824 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@7cb8be name:ZooKeeperConnection Watcher:127.0.0.1:58752/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 692982 T3813 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 692982 T3787 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 692982 T3821 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 692983 T3821 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 692983 T3814 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection"} current state version: 1
   [junit4]   2> 692983 T3787 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> ASYNC  NEW_CORE C553 name=collection1 org.apache.solr.core.SolrCore@606160 url=http://127.0.0.1:56919/collection1 node=127.0.0.1:56919_ C553_STATE=coll:control_collection core:collection1 props:{base_url=http://127.0.0.1:56919, core=collection1, node_name=127.0.0.1:56919_, state=down}
   [junit4]   2> 692983 T3821 C553 P56919 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:56919/collection1/
   [junit4]   2> 692985 T3821 C553 P56919 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 692985 T3787 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 692985 T3821 C553 P56919 oasc.SyncStrategy.syncToMe http://127.0.0.1:56919/collection1/ has no replicas
   [junit4]   2> 692986 T3821 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:56919/collection1/ shard1
   [junit4]   2> 692986 T3821 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 692987 T3787 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> 692989 T3813 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 692990 T3814 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:56919",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active"} current state version: 1
   [junit4]   2> 693092 T3813 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 693092 T3824 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 693140 T3821 oasc.ZkController.register We are http://127.0.0.1:56919/collection1/ and leader is http://127.0.0.1:56919/collection1/
   [junit4]   2> 693141 T3821 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:56919
   [junit4]   2> 693141 T3821 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 693141 T3821 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
   [junit4]   2> 693141 T3821 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 693143 T3813 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 693144 T3814 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "base_url":"http://127.0.0.1:56919",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "node_name":"127.0.0.1:56919_",
   [junit4]   2> 	  "state":"active"} current state version: 2
   [junit4]   2> 693145 T3814 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "base_url":"http://127.0.0.1:56919",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "node_name":"127.0.0.1:56919_",
   [junit4]   2> 	  "state":"active"}
   [junit4]   2> 693248 T3813 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 693248 T3824 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 693264 T3787 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 693265 T3787 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 693266 T3787 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:?????
   [junit4]   2> 693267 T3787 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@47034b
   [junit4]   2> 693267 T3787 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 693268 T3787 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-003
   [junit4]   2> 693268 T3787 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-003/'
   [junit4]   2> 693286 T3787 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-003/solr.xml
   [junit4]   2> 693319 T3787 oasc.CoreContainer.<init> New CoreContainer 31037377
   [junit4]   2> 693319 T3787 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-003/]
   [junit4]   2> 693320 T3787 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 693321 T3787 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 693321 T3787 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 693321 T3787 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 693322 T3787 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 693322 T3787 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 693322 T3787 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 693323 T3787 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 693323 T3787 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 693324 T3787 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 693324 T3787 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 693325 T3787 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 693325 T3787 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 693325 T3787 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 693326 T3787 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58752/solr
   [junit4]   2> 693326 T3787 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 693326 T3787 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 693327 T3787 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 693329 T3836 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@239e96 name:ZooKeeperConnection Watcher:127.0.0.1:58752 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 693329 T3787 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 693330 T3787 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 693331 T3787 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 693332 T3839 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@f0e5f7 name:ZooKeeperConnection Watcher:127.0.0.1:58752/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 693333 T3787 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 693337 T3787 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 694340 T3787 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57197_
   [junit4]   2> 694341 T3787 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57197_
   [junit4]   2> 694344 T3787 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 694349 T3840 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 694350 T3840 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 694351 T3840 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 694351 T3813 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 694352 T3814 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "base_url":"http://127.0.0.1:57197",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "node_name":"127.0.0.1:57197_",
   [junit4]   2> 	  "state":"down"} current state version: 3
   [junit4]   2> 694352 T3814 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "base_url":"http://127.0.0.1:57197",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "node_name":"127.0.0.1:57197_",
   [junit4]   2> 	  "state":"down"}
   [junit4]   2> 694353 T3814 oasco.ClusterStateMutator.createCollection building a new cName: collection1
   [junit4]   2> 694353 T3814 oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 694456 T3813 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 694456 T3839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 694456 T3824 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 695351 T3840 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 695351 T3840 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 695352 T3840 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 695352 T3840 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 695353 T3840 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 695353 T3840 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-003/collection1/'
   [junit4]   2> 695354 T3840 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading%20A5CA2100C118E8DA-001/tempDir-003/collection1/lib/.svn/' to classloader
   [junit4]   2> 695355 T3840 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading%20A5CA2100C118E8DA-001/tempDir-003/collection1/lib/classes/' to classloader
   [junit4]   2> 695355 T3840 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading%20A5CA2100C118E8DA-001/tempDir-003/collection1/lib/README' to classloader
   [junit4]   2> 695372 T3840 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 695374 T3840 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 695380 T3840 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 695395 T3840 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 695426 T3840 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 695428 T3840 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 695436 T3840 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 695704 T3840 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 695709 T3840 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 695712 T3840 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 695721 T3840 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 695724 T3840 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 695726 T3840 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 695727 T3840 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 695728 T3840 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 695728 T3840 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 695729 T3840 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 695729 T3840 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 695729 T3840 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 695729 T3840 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 695730 T3840 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-003/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-001/jetty1/
   [junit4]   2> 695730 T3840 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@17ba1a8
   [junit4]   2> 695730 T3840 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-001/jetty1
   [junit4]   2> 695731 T3840 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-001/jetty1/index/
   [junit4]   2> 695731 T3840 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
   [junit4]   2> 695731 T3840 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-001/jetty1/index
   [junit4]   2> 695732 T3840 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=34, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 695732 T3840 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@15ece3b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@135a3fd),segFN=segments_1,generation=1}
   [junit4]   2> 695733 T3840 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 695736 T3840 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 695736 T3840 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 695737 T3840 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 695737 T3840 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 695737 T3840 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 695737 T3840 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 695738 T3840 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 695738 T3840 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 695738 T3840 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 695739 T3840 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 695739 T3840 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 695740 T3840 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 695740 T3840 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 695741 T3840 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
   [junit4]   2> 695742 T3840 oasc.RequestHandlers.initHandlersFromConfig created /schema: org.apache.solr.handler.SchemaHandler
   [junit4]   2> 695742 T3840 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler
   [junit4]   2> 695743 T3840 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler
   [junit4]   2> 695744 T3840 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
   [junit4]   2> 695745 T3840 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
   [junit4]   2> 695745 T3840 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
   [junit4]   2> 695746 T3840 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
   [junit4]   2> 695746 T3840 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
   [junit4]   2> 695747 T3840 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler
   [junit4]   2> 695748 T3840 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
   [junit4]   2> 695748 T3840 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler
   [junit4]   2> 695749 T3840 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 695750 T3840 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 695752 T3840 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 695753 T3840 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 695755 T3840 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 695756 T3840 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 695757 T3840 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 695758 T3840 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 695758 T3840 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 695759 T3840 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@1581ccb
   [junit4]   2> 695759 T3840 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@15ece3b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@135a3fd),segFN=segments_1,generation=1}
   [junit4]   2> 695760 T3840 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 695760 T3840 oass.SolrIndexSearcher.<init> Opening Searcher@1740a37[collection1] main
   [junit4]   2> 695760 T3840 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 695761 T3840 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 695761 T3840 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 695762 T3840 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 695762 T3840 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 695762 T3840 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 695763 T3840 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 695763 T3840 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 695763 T3840 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 695763 T3840 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 695764 T3841 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1740a37[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 695765 T3840 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 695766 T3844 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57197 collection:collection1 shard:shard1
   [junit4]   2> 695766 T3787 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 695766 T3844 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 695766 T3787 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 695770 T3844 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 695771 T3813 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 695772 T3844 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 695772 T3844 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 695772 T3814 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1"} current state version: 4
   [junit4]   2> ASYNC  NEW_CORE C554 name=collection1 org.apache.solr.core.SolrCore@1781aba url=http://127.0.0.1:57197/collection1 node=127.0.0.1:57197_ C554_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:57197, core=collection1, node_name=127.0.0.1:57197_, state=down}
   [junit4]   2> 695772 T3844 C554 P57197 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57197/collection1/
   [junit4]   2> 695773 T3844 C554 P57197 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 695773 T3844 C554 P57197 oasc.SyncStrategy.syncToMe http://127.0.0.1:57197/collection1/ has no replicas
   [junit4]   2> 695773 T3844 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:57197/collection1/ shard1
   [junit4]   2> 695774 T3844 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 695777 T3813 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 695778 T3814 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:57197",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active"} current state version: 4
   [junit4]   2> 695880 T3824 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 695880 T3839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 695881 T3813 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 695928 T3844 oasc.ZkController.register We are http://127.0.0.1:57197/collection1/ and leader is http://127.0.0.1:57197/collection1/
   [junit4]   2> 695928 T3844 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:57197
   [junit4]   2> 695929 T3844 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 695929 T3844 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 695929 T3844 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 695930 T3813 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 695931 T3814 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "base_url":"http://127.0.0.1:57197",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "node_name":"127.0.0.1:57197_",
   [junit4]   2> 	  "state":"active"} current state version: 5
   [junit4]   2> 695932 T3814 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "base_url":"http://127.0.0.1:57197",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "node_name":"127.0.0.1:57197_",
   [junit4]   2> 	  "state":"active"}
   [junit4]   2> 696035 T3813 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 696035 T3839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 696035 T3824 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 696045 T3787 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 696046 T3787 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 696047 T3787 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:?????
   [junit4]   2> 696048 T3787 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@47034b
   [junit4]   2> 696048 T3787 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 696049 T3787 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-004
   [junit4]   2> 696049 T3787 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-004/'
   [junit4]   2> 696067 T3787 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-004/solr.xml
   [junit4]   2> 696100 T3787 oasc.CoreContainer.<init> New CoreContainer 24380689
   [junit4]   2> 696100 T3787 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-004/]
   [junit4]   2> 696101 T3787 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 696102 T3787 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 696102 T3787 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 696102 T3787 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 696103 T3787 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 696103 T3787 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 696103 T3787 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 696104 T3787 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 696104 T3787 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 696104 T3787 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 696105 T3787 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 696105 T3787 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 696106 T3787 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 696106 T3787 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 696107 T3787 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:58752/solr
   [junit4]   2> 696107 T3787 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 696107 T3787 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 696108 T3787 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 696109 T3856 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8de48b name:ZooKeeperConnection Watcher:127.0.0.1:58752 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 696111 T3787 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 696111 T3787 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 696114 T3787 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 696115 T3859 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1027b41 name:ZooKeeperConnection Watcher:127.0.0.1:58752/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 696115 T3787 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 696119 T3787 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 697122 T3787 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:57847_
   [junit4]   2> 697124 T3787 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:57847_
   [junit4]   2> 697127 T3787 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 697132 T3860 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 697132 T3860 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 697133 T3860 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 697133 T3813 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 697134 T3814 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "base_url":"http://127.0.0.1:57847",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "node_name":"127.0.0.1:57847_",
   [junit4]   2> 	  "state":"down"} current state version: 6
   [junit4]   2> 697135 T3814 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "base_url":"http://127.0.0.1:57847",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "node_name":"127.0.0.1:57847_",
   [junit4]   2> 	  "state":"down"}
   [junit4]   2> 697135 T3814 oasco.ReplicaMutator.updateState Collection already exists with numShards=2
   [junit4]   2> 697135 T3814 oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 697238 T3824 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 697238 T3839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 697238 T3813 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 697238 T3859 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 698133 T3860 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 698134 T3860 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 698135 T3860 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 698135 T3860 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 698136 T3860 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 698136 T3860 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-004/collection1/'
   [junit4]   2> 698137 T3860 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading%20A5CA2100C118E8DA-001/tempDir-004/collection1/lib/.svn/' to classloader
   [junit4]   2> 698138 T3860 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading%20A5CA2100C118E8DA-001/tempDir-004/collection1/lib/classes/' to classloader
   [junit4]   2> 698138 T3860 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading%20A5CA2100C118E8DA-001/tempDir-004/collection1/lib/README' to classloader
   [junit4]   2> 698163 T3860 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 698165 T3860 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 698174 T3860 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 698197 T3860 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 698245 T3860 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 698247 T3860 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 698261 T3860 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 698600 T3860 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 698606 T3860 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 698608 T3860 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 698616 T3860 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 698619 T3860 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 698622 T3860 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 698623 T3860 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 698623 T3860 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 698624 T3860 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 698624 T3860 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 698625 T3860 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 698625 T3860 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 698625 T3860 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 698625 T3860 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-004/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-001/jetty2/
   [junit4]   2> 698626 T3860 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@17ba1a8
   [junit4]   2> 698627 T3860 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-001/jetty2
   [junit4]   2> 698627 T3860 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-001/jetty2/index/
   [junit4]   2> 698627 T3860 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-001/jetty2/index' doesn't exist. Creating new index...
   [junit4]   2> 698627 T3860 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-001/jetty2/index
   [junit4]   2> 698628 T3860 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogDocMergePolicy: [LogDocMergePolicy: minMergeSize=1000, mergeFactor=34, maxMergeSize=9223372036854775807, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 698628 T3860 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@146dc8b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1392c61),segFN=segments_1,generation=1}
   [junit4]   2> 698629 T3860 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 698632 T3860 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 698632 T3860 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 698633 T3860 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 698633 T3860 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 698633 T3860 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 698633 T3860 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 698634 T3860 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 698634 T3860 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 698634 T3860 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 698635 T3860 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 698635 T3860 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 698636 T3860 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 698636 T3860 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 698637 T3860 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
   [junit4]   2> 698638 T3860 oasc.RequestHandlers.initHandlersFromConfig created /schema: org.apache.solr.handler.SchemaHandler
   [junit4]   2> 698638 T3860 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler
   [junit4]   2> 698639 T3860 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler
   [junit4]   2> 698640 T3860 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
   [junit4]   2> 698640 T3860 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
   [junit4]   2> 698641 T3860 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
   [junit4]   2> 698642 T3860 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
   [junit4]   2> 698642 T3860 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
   [junit4]   2> 698643 T3860 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler
   [junit4]   2> 698644 T3860 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
   [junit4]   2> 698644 T3860 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler
   [junit4]   2> 698645 T3860 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 698645 T3860 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 698648 T3860 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 698649 T3860 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 698650 T3860 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 698651 T3860 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 698653 T3860 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 698654 T3860 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 698654 T3860 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 698654 T3860 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.MockRandomMergePolicy: org.apache.lucene.index.MockRandomMergePolicy@1e8ff72
   [junit4]   2> 698655 T3860 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@146dc8b lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1392c61),segFN=segments_1,generation=1}
   [junit4]   2> 698655 T3860 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 698655 T3860 oass.SolrIndexSearcher.<init> Opening Searcher@1345817[collection1] main
   [junit4]   2> 698656 T3860 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 698657 T3860 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 698657 T3860 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 698657 T3860 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 698658 T3860 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 698658 T3860 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 698658 T3860 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 698659 T3860 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 698659 T3860 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 698659 T3860 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 698660 T3861 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1345817[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 698661 T3860 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 698661 T3864 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:57847 collection:collection1 shard:shard2
   [junit4]   2> 698662 T3787 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1
   [junit4]   2> 698662 T3787 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 698662 T3864 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 698667 T3864 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 698668 T3813 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 698668 T3864 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 698668 T3864 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C555 name=collection1 org.apache.solr.core.SolrCore@17680be url=http://127.0.0.1:57847/collection1 node=127.0.0.1:57847_ C555_STATE=coll:collection1 core:collection1 props:{base_url=http://127.0.0.1:57847, core=collection1, node_name=127.0.0.1:57847_, state=down}
   [junit4]   2> 698669 T3864 C555 P57847 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:57847/collection1/
   [junit4]   2> 698669 T3814 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1"} current state version: 7
   [junit4]   2> 698669 T3864 C555 P57847 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 698669 T3864 C555 P57847 oasc.SyncStrategy.syncToMe http://127.0.0.1:57847/collection1/ has no replicas
   [junit4]   2> 698670 T3864 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:57847/collection1/ shard2
   [junit4]   2> 698670 T3864 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 698672 T3813 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 698673 T3814 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:57847",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active"} current state version: 7
   [junit4]   2> 698775 T3824 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 698775 T3839 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 698775 T3859 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 698775 T3813 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 698823 T3864 oasc.ZkController.register We are http://127.0.0.1:57847/collection1/ and leader is http://127.0.0.1:57847/collection1/
   [junit4]   2> 698823 T3864 oasc.ZkController.register No Log

[...truncated too long message...]

e [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@928804
   [junit4]   2> 717682 T3787 oasc.ZkController.unRegisterConfListener  a listener was removed because of core close
   [junit4]   2> 717682 T3957 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "base_url":"http://127.0.0.1:58529",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node4",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "node_name":"127.0.0.1:58529_",
   [junit4]   2> 	  "state":"down"} current state version: 30
   [junit4]   2> 717682 T3787 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
   [junit4]   2> 717683 T3957 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "base_url":"http://127.0.0.1:58529",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "operation":"state",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "core_node_name":"core_node4",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "node_name":"127.0.0.1:58529_",
   [junit4]   2> 	  "state":"down"}
   [junit4]   2> 717683 T3787 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 717684 T3787 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 717684 T3787 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 717685 T3787 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 717699 T3787 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 717700 T3787 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-001/jetty4/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-001/jetty4/index;done=false>>]
   [junit4]   2> 717700 T3787 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-001/jetty4/index
   [junit4]   2> 717700 T3787 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-001/jetty4 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-001/jetty4;done=false>>]
   [junit4]   2> 717701 T3787 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001/tempDir-001/jetty4
   [junit4]   2> 717702 T3787 oasc.Overseer.close Overseer (id=93110379544969228-127.0.0.1:58529_-n_0000000004) closing
   [junit4]   2> 717702 T3957 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:58529_
   [junit4]   2> 719203 T3950 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 719225 T3787 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
   [junit4]   2> 719415 T3787 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
   [junit4]   2> 719417 T3787 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:58752 58752
   [junit4]   2> 719958 T3788 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:58752 58752
   [junit4]   2> 719960 T3788 oasc.ZkTestServer$ZKServerMain.runFromConfig WARN Watch limit violations: 
   [junit4]   2> 	Maximum concurrent create/delete watches above limit:
   [junit4]   2> 	
   [junit4]   2> 		10	/solr/configs/conf1
   [junit4]   2> 		6	/solr/clusterstate.json
   [junit4]   2> 		6	/solr/aliases.json
   [junit4]   2> 		2	/solr/configs/.system
   [junit4]   2> 	
   [junit4]   2> 	Maximum concurrent children watches above limit:
   [junit4]   2> 	
   [junit4]   2> 		6	/solr/live_nodes
   [junit4]   2> 		5	/solr/overseer/queue
   [junit4]   2> 		5	/solr/overseer/collection-queue-work
   [junit4]   2> 	
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestDynamicLoading -Dtests.method=testDistribSearch -Dtests.seed=A5CA2100C118E8DA -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=hi_IN -Dtests.timezone=GB-Eire -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
   [junit4] FAILURE 29.1s J1 | TestDynamicLoading.testDistribSearch <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: {
   [junit4]    >   "responseHeader":{
   [junit4]    >     "status":404,
   [junit4]    >     "QTime":3},
   [junit4]    >   "error":{
   [junit4]    >     "msg":"no such blob or version available: test/1",
   [junit4]    >     "code":404}}
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([A5CA2100C118E8DA:242CAF18B64788E6]:0)
   [junit4]    > 	at org.apache.solr.core.TestDynamicLoading.dynamicLoading(TestDynamicLoading.java:108)
   [junit4]    > 	at org.apache.solr.core.TestDynamicLoading.doTest(TestDynamicLoading.java:64)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:868)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 719967 T3787 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J1/temp/solr.core.TestDynamicLoading A5CA2100C118E8DA-001
   [junit4]   2> NOTE: test params are: codec=Lucene50, sim=DefaultSimilarity, locale=hi_IN, timezone=GB-Eire
   [junit4]   2> NOTE: Linux 3.13.0-43-generic i386/Oracle Corporation 1.7.0_72 (32-bit)/cpus=8,threads=1,free=224027952,total=536870912
   [junit4]   2> NOTE: All tests run in this JVM: [HighlighterTest, SchemaVersionSpecificBehaviorTest, TermsComponentTest, TestSerializedLuceneMatchVersion, TestChildDocTransformer, TestElisionMultitermQuery, AutoCommitTest, TestFieldTypeCollectionResource, TestPivotHelperCode, TestSolr4Spatial, JsonLoaderTest, TestSearcherReuse, BasicZkTest, SolrCmdDistributorTest, URLClassifyProcessorTest, ShardRoutingCustomTest, IndexBasedSpellCheckerTest, TestFieldResource, TestDefaultSimilarityFactory, DisMaxRequestHandlerTest, TestImplicitCoreProperties, TestLuceneMatchVersion, DistanceUnitsTest, TestZkChroot, SearchHandlerTest, PrimitiveFieldTypeTest, HttpPartitionTest, OverseerRolesTest, SimplePostToolTest, MBeansHandlerTest, TestSuggestSpellingConverter, TestDefaultSearchFieldResource, SpellingQueryConverterTest, DocValuesMissingTest, TestUtils, TestStressUserVersions, BadIndexSchemaTest, TestPseudoReturnFields, PolyFieldTest, HdfsBasicDistributedZk2Test, SynonymTokenizerTest, HdfsSyncSliceTest, TestInitQParser, IndexSchemaTest, DeleteLastCustomShardedReplicaTest, AnalyticsQueryTest, TestSolrXmlPersistor, OverseerStatusTest, TermVectorComponentTest, TestRawResponseWriter, ZkSolrClientTest, CloudMLTQParserTest, TestNRTOpen, UpdateRequestProcessorFactoryTest, TestSimpleQParserPlugin, TestBM25SimilarityFactory, TestJoin, TestMaxScoreQueryParser, HdfsBasicDistributedZkTest, SpellCheckComponentTest, WordBreakSolrSpellCheckerTest, CollectionsAPIDistributedZkTest, EchoParamsTest, OverseerCollectionProcessorTest, TestComplexPhraseQParserPlugin, TestBulkSchemaAPI, TestReversedWildcardFilterFactory, TestManagedSchemaFieldResource, FileUtilsTest, HdfsChaosMonkeySafeLeaderTest, TestDistributedGrouping, TestRecovery, ChaosMonkeyNothingIsSafeTest, BasicFunctionalityTest, CursorPagingTest, LoggingHandlerTest, TestAnalyzeInfixSuggestions, ResponseLogComponentTest, HdfsRecoveryZkTest, TestDynamicLoading]
   [junit4] Completed on J1 in 29.16s, 1 test, 1 failure <<< FAILURES!

[...truncated 942 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:529: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:477: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:61: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/extra-targets.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build.xml:190: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/common-build.xml:508: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:1363: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:970: There were test failures: 459 suites, 1869 tests, 1 failure, 47 ignored (22 assumptions)

Total time: 77 minutes 42 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 32bit/jdk1.7.0_72 -client -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-5.x-Linux (32bit/jdk1.8.0_40-ea-b20) - Build # 11383 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/11383/
Java: 32bit/jdk1.8.0_40-ea-b20 -server -XX:+UseSerialGC

1 tests failed.
FAILED:  org.apache.solr.handler.TestSolrConfigHandlerCloud.testDistribSearch

Error Message:
Could not get expected value  CY val modified for path [response, params, y, c] full output {   "responseHeader":{     "status":0,     "QTime":0},   "response":{     "znodeVersion":0,     "params":{"x":{         "a":"A val",         "b":"B val",         "":{"v":0}}}}}

Stack Trace:
java.lang.AssertionError: Could not get expected value  CY val modified for path [response, params, y, c] full output {
  "responseHeader":{
    "status":0,
    "QTime":0},
  "response":{
    "znodeVersion":0,
    "params":{"x":{
        "a":"A val",
        "b":"B val",
        "":{"v":0}}}}}
	at __randomizedtesting.SeedInfo.seed([311931BA026040D2:B0FFBFA2753F20EE]:0)
	at org.junit.Assert.fail(Assert.java:93)
	at org.junit.Assert.assertTrue(Assert.java:43)
	at org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:256)
	at org.apache.solr.handler.TestSolrConfigHandlerCloud.testReqParams(TestSolrConfigHandlerCloud.java:225)
	at org.apache.solr.handler.TestSolrConfigHandlerCloud.doTest(TestSolrConfigHandlerCloud.java:61)
	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:868)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:497)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesRestoreRule$1.evaluate(SystemPropertiesRestoreRule.java:53)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)




Build Log:
[...truncated 8860 lines...]
   [junit4] Suite: org.apache.solr.handler.TestSolrConfigHandlerCloud
   [junit4]   2> Creating dataDir: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/init-core-data-001
   [junit4]   2> 97980 T426 oas.SolrTestCaseJ4.buildSSLConfig Randomized ssl (false) and clientAuth (false)
   [junit4]   2> 97980 T426 oas.BaseDistributedSearchTestCase.initHostContext Setting hostContext system property: /
   [junit4]   2> 97983 T426 oas.SolrTestCaseJ4.setUp ###Starting testDistribSearch
   [junit4]   2> 97983 T426 oasc.ZkTestServer.run STARTING ZK TEST SERVER
   [junit4]   1> client port:0.0.0.0/0.0.0.0:0
   [junit4]   2> 97984 T427 oasc.ZkTestServer$ZKServerMain.runFromConfig Starting server
   [junit4]   2> 98084 T426 oasc.ZkTestServer.run start zk server on port:47620
   [junit4]   2> 98084 T426 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 98085 T426 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 98089 T434 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@e74805 name:ZooKeeperConnection Watcher:127.0.0.1:47620 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 98089 T426 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 98089 T426 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 98090 T426 oascc.SolrZkClient.makePath makePath: /solr
   [junit4]   2> 98093 T426 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 98093 T426 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 98094 T437 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@69beaf name:ZooKeeperConnection Watcher:127.0.0.1:47620/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 98095 T426 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 98095 T426 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 98095 T426 oascc.SolrZkClient.makePath makePath: /collections/collection1
   [junit4]   2> 98096 T426 oascc.SolrZkClient.makePath makePath: /collections/collection1/shards
   [junit4]   2> 98098 T426 oascc.SolrZkClient.makePath makePath: /collections/control_collection
   [junit4]   2> 98099 T426 oascc.SolrZkClient.makePath makePath: /collections/control_collection/shards
   [junit4]   2> 98100 T426 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig-tlog.xml to /configs/conf1/solrconfig.xml
   [junit4]   2> 98100 T426 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.xml
   [junit4]   2> 98102 T426 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/schema.xml to /configs/conf1/schema.xml
   [junit4]   2> 98102 T426 oascc.SolrZkClient.makePath makePath: /configs/conf1/schema.xml
   [junit4]   2> 98104 T426 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/solrconfig.snippet.randomindexconfig.xml to /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 98104 T426 oascc.SolrZkClient.makePath makePath: /configs/conf1/solrconfig.snippet.randomindexconfig.xml
   [junit4]   2> 98105 T426 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/stopwords.txt to /configs/conf1/stopwords.txt
   [junit4]   2> 98105 T426 oascc.SolrZkClient.makePath makePath: /configs/conf1/stopwords.txt
   [junit4]   2> 98107 T426 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/protwords.txt to /configs/conf1/protwords.txt
   [junit4]   2> 98107 T426 oascc.SolrZkClient.makePath makePath: /configs/conf1/protwords.txt
   [junit4]   2> 98108 T426 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/currency.xml to /configs/conf1/currency.xml
   [junit4]   2> 98108 T426 oascc.SolrZkClient.makePath makePath: /configs/conf1/currency.xml
   [junit4]   2> 98109 T426 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/enumsConfig.xml to /configs/conf1/enumsConfig.xml
   [junit4]   2> 98110 T426 oascc.SolrZkClient.makePath makePath: /configs/conf1/enumsConfig.xml
   [junit4]   2> 98111 T426 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/open-exchange-rates.json to /configs/conf1/open-exchange-rates.json
   [junit4]   2> 98111 T426 oascc.SolrZkClient.makePath makePath: /configs/conf1/open-exchange-rates.json
   [junit4]   2> 98112 T426 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/mapping-ISOLatin1Accent.txt to /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 98113 T426 oascc.SolrZkClient.makePath makePath: /configs/conf1/mapping-ISOLatin1Accent.txt
   [junit4]   2> 98114 T426 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/old_synonyms.txt to /configs/conf1/old_synonyms.txt
   [junit4]   2> 98114 T426 oascc.SolrZkClient.makePath makePath: /configs/conf1/old_synonyms.txt
   [junit4]   2> 98115 T426 oasc.AbstractZkTestCase.putConfig put /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/core/src/test-files/solr/collection1/conf/synonyms.txt to /configs/conf1/synonyms.txt
   [junit4]   2> 98115 T426 oascc.SolrZkClient.makePath makePath: /configs/conf1/synonyms.txt
   [junit4]   2> 98365 T426 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 98369 T426 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:33649
   [junit4]   2> 98369 T426 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@e2f2a
   [junit4]   2> 98369 T426 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 98370 T426 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-002
   [junit4]   2> 98370 T426 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-002/'
   [junit4]   2> 98393 T426 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-002/solr.xml
   [junit4]   2> 98401 T426 oasc.CoreContainer.<init> New CoreContainer 33218090
   [junit4]   2> 98402 T426 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-002/]
   [junit4]   2> 98402 T426 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 98403 T426 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 98403 T426 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 98403 T426 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 98403 T426 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 98404 T426 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 98404 T426 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 98404 T426 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 98404 T426 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 98405 T426 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 98405 T426 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 98406 T426 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 98406 T426 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 98406 T426 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 98407 T426 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47620/solr
   [junit4]   2> 98407 T426 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 98407 T426 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 98408 T426 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 98416 T449 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@13dc4d7 name:ZooKeeperConnection Watcher:127.0.0.1:47620 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 98416 T426 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 98417 T426 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 98419 T426 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 98421 T452 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@92c2d8 name:ZooKeeperConnection Watcher:127.0.0.1:47620/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 98421 T426 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 98422 T426 oascc.SolrZkClient.makePath makePath: /overseer/queue
   [junit4]   2> 98424 T426 oascc.SolrZkClient.makePath makePath: /overseer/collection-queue-work
   [junit4]   2> 98426 T426 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-running
   [junit4]   2> 98428 T426 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-completed
   [junit4]   2> 98430 T426 oascc.SolrZkClient.makePath makePath: /overseer/collection-map-failure
   [junit4]   2> 98432 T426 oascc.SolrZkClient.makePath makePath: /live_nodes
   [junit4]   2> 98433 T426 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:33649_
   [junit4]   2> 98434 T426 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:33649_
   [junit4]   2> 98435 T426 oascc.SolrZkClient.makePath makePath: /overseer_elect
   [junit4]   2> 98437 T426 oascc.SolrZkClient.makePath makePath: /overseer_elect/election
   [junit4]   2> 98438 T426 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 98439 T426 oasc.OverseerElectionContext.runLeaderProcess I am going to be the leader 127.0.0.1:33649_
   [junit4]   2> 98440 T426 oascc.SolrZkClient.makePath makePath: /overseer_elect/leader
   [junit4]   2> 98441 T426 oasc.Overseer.start Overseer (id=93109939143245827-127.0.0.1:33649_-n_0000000000) starting
   [junit4]   2> 98443 T426 oascc.SolrZkClient.makePath makePath: /overseer/queue-work
   [junit4]   2> 98449 T426 oasc.OverseerAutoReplicaFailoverThread.<init> Starting OverseerAutoReplicaFailoverThread autoReplicaFailoverWorkLoopDelay=10000 autoReplicaFailoverWaitAfterExpiration=30000 autoReplicaFailoverBadNodeExpiration=60000
   [junit4]   2> 98449 T454 oasc.OverseerCollectionProcessor.run Process current queue of collection creations
   [junit4]   2> 98450 T426 oascc.SolrZkClient.makePath makePath: /clusterstate.json
   [junit4]   2> 98453 T426 oascc.SolrZkClient.makePath makePath: /aliases.json
   [junit4]   2> 98454 T426 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 98457 T453 oasc.Overseer$ClusterStateUpdater.run Starting to work on the main queue
   [junit4]   2> 98459 T456 oasc.ZkController.publish publishing core=collection1 state=down collection=control_collection
   [junit4]   2> 98459 T456 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 98460 T452 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 98460 T456 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 98463 T453 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 0, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:33649",
   [junit4]   2> 	  "node_name":"127.0.0.1:33649_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "operation":"state"} current state version: 0
   [junit4]   2> 98463 T453 oasco.ReplicaMutator.updateState Update state numShards=1 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:33649",
   [junit4]   2> 	  "node_name":"127.0.0.1:33649_",
   [junit4]   2> 	  "numShards":"1",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 98463 T453 oasco.ClusterStateMutator.createCollection building a new cName: control_collection
   [junit4]   2> 98464 T453 oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 98465 T452 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 99461 T456 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 99461 T456 oasc.ZkController.createCollectionZkNode Check for collection zkNode:control_collection
   [junit4]   2> 99462 T456 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 99462 T456 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 99463 T456 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 99463 T456 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-002/collection1/'
   [junit4]   2> 99464 T456 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud%20311931BA026040D2-001/tempDir-002/collection1/lib/.svn/' to classloader
   [junit4]   2> 99464 T456 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud%20311931BA026040D2-001/tempDir-002/collection1/lib/classes/' to classloader
   [junit4]   2> 99464 T456 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud%20311931BA026040D2-001/tempDir-002/collection1/lib/README' to classloader
   [junit4]   2> 99477 T456 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 99479 T456 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 99485 T456 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 99491 T456 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 99503 T456 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 99505 T456 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 99509 T456 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 99621 T456 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 99625 T456 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 99626 T456 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 99635 T456 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 99638 T456 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 99640 T456 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 99641 T456 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 99641 T456 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 99642 T456 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 99643 T456 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 99643 T456 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 99643 T456 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection control_collection
   [junit4]   2> 99643 T456 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 99644 T456 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-002/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-001/control/data/
   [junit4]   2> 99644 T456 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@11deaa5
   [junit4]   2> 99645 T456 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-001/control/data
   [junit4]   2> 99645 T456 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-001/control/data/index/
   [junit4]   2> 99645 T456 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-001/control/data/index' doesn't exist. Creating new index...
   [junit4]   2> 99646 T456 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-001/control/data/index
   [junit4]   2> 99646 T456 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=21, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 99647 T456 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@15fe459 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@123689e),segFN=segments_1,generation=1}
   [junit4]   2> 99647 T456 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 99651 T456 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 99652 T456 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 99652 T456 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 99652 T456 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 99653 T456 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 99653 T456 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 99653 T456 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 99654 T456 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 99654 T456 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 99655 T456 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 99655 T456 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 99655 T456 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 99656 T456 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 99656 T456 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
   [junit4]   2> 99657 T456 oasc.RequestHandlers.initHandlersFromConfig created /schema: org.apache.solr.handler.SchemaHandler
   [junit4]   2> 99657 T456 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler
   [junit4]   2> 99657 T456 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler
   [junit4]   2> 99658 T456 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
   [junit4]   2> 99658 T456 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
   [junit4]   2> 99659 T456 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
   [junit4]   2> 99659 T456 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
   [junit4]   2> 99659 T456 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
   [junit4]   2> 99660 T456 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler
   [junit4]   2> 99660 T456 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
   [junit4]   2> 99660 T456 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler
   [junit4]   2> 99660 T456 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 99661 T456 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 99663 T456 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 99663 T456 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 99664 T456 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 99665 T456 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 99666 T456 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 99667 T456 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 99668 T456 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 99668 T456 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=32, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=67.88671875, floorSegmentMB=0.5439453125, forceMergeDeletesPctAllowed=2.3235970957651753, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 99669 T456 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@15fe459 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@123689e),segFN=segments_1,generation=1}
   [junit4]   2> 99669 T456 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 99670 T456 oass.SolrIndexSearcher.<init> Opening Searcher@1c3e3e9[collection1] main
   [junit4]   2> 99670 T456 oascc.ZkStateReader.readConfigName Load collection config from:/collections/control_collection
   [junit4]   2> 99671 T456 oascc.ZkStateReader.readConfigName path=/collections/control_collection configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 99671 T456 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 99672 T456 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 99672 T456 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 99672 T456 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 99673 T456 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 99673 T456 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 99673 T456 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 99673 T456 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 99674 T457 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1c3e3e9[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 99674 T456 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 99675 T460 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:33649 collection:control_collection shard:shard1
   [junit4]   2> 99675 T426 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 99676 T426 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 99676 T460 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leader_elect/shard1/election
   [junit4]   2> 99676 T426 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 99677 T426 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 99680 T463 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@eacba9 name:ZooKeeperConnection Watcher:127.0.0.1:47620/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 99681 T426 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 99681 T426 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 99682 T426 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 99683 T460 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 99684 T452 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 99684 T460 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 99684 T460 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> 99684 T426 oasc.ChaosMonkey.monkeyLog monkey: init - expire sessions:false cause connection loss:false
   [junit4]   2> ASYNC  NEW_CORE C167 name=collection1 org.apache.solr.core.SolrCore@e0ccf7 url=http://127.0.0.1:33649/collection1 node=127.0.0.1:33649_ C167_STATE=coll:control_collection core:collection1 props:{core=collection1, base_url=http://127.0.0.1:33649, node_name=127.0.0.1:33649_, state=down}
   [junit4]   2> 99684 T460 C167 P33649 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:33649/collection1/
   [junit4]   2> 99685 T460 C167 P33649 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 99685 T460 C167 P33649 oasc.SyncStrategy.syncToMe http://127.0.0.1:33649/collection1/ has no replicas
   [junit4]   2> 99686 T460 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:33649/collection1/ shard1
   [junit4]   2> 99686 T460 oascc.SolrZkClient.makePath makePath: /collections/control_collection/leaders/shard1
   [junit4]   2> 99686 T453 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection"} current state version: 1
   [junit4]   2> 99689 T452 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 99690 T453 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:33649",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active"} current state version: 1
   [junit4]   2> 99792 T452 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 99792 T463 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 99840 T460 oasc.ZkController.register We are http://127.0.0.1:33649/collection1/ and leader is http://127.0.0.1:33649/collection1/
   [junit4]   2> 99840 T460 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:33649
   [junit4]   2> 99840 T460 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 99841 T460 oasc.ZkController.publish publishing core=collection1 state=active collection=control_collection
   [junit4]   2> 99841 T460 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 99842 T452 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 99843 T453 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:33649",
   [junit4]   2> 	  "node_name":"127.0.0.1:33649_",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "operation":"state"} current state version: 2
   [junit4]   2> 99844 T453 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:33649",
   [junit4]   2> 	  "node_name":"127.0.0.1:33649_",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"control_collection",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 99933 T426 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 1
   [junit4]   2> 99933 T426 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 99935 T426 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:44431
   [junit4]   2> 99936 T426 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@e2f2a
   [junit4]   2> 99936 T426 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 99937 T426 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-003
   [junit4]   2> 99937 T426 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-003/'
   [junit4]   2> 99946 T463 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 99946 T452 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 99949 T426 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-003/solr.xml
   [junit4]   2> 99957 T426 oasc.CoreContainer.<init> New CoreContainer 21620026
   [junit4]   2> 99957 T426 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-003/]
   [junit4]   2> 99957 T426 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 99958 T426 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 99958 T426 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 99958 T426 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 99958 T426 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 99959 T426 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 99959 T426 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 99959 T426 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 99959 T426 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 99960 T426 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 99960 T426 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 99960 T426 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 99961 T426 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 99961 T426 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 99961 T426 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47620/solr
   [junit4]   2> 99961 T426 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 99962 T426 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 99962 T426 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 99964 T475 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@19d524f name:ZooKeeperConnection Watcher:127.0.0.1:47620 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 99964 T426 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 99965 T426 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 99966 T426 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 99968 T478 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1360d99 name:ZooKeeperConnection Watcher:127.0.0.1:47620/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 99968 T426 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 99973 T426 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 100976 T426 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:44431_
   [junit4]   2> 100977 T426 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:44431_
   [junit4]   2> 100979 T426 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 100982 T479 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 100983 T479 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 100983 T452 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 100983 T479 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 100984 T453 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:44431",
   [junit4]   2> 	  "node_name":"127.0.0.1:44431_",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"} current state version: 3
   [junit4]   2> 100985 T453 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:44431",
   [junit4]   2> 	  "node_name":"127.0.0.1:44431_",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 100985 T453 oasco.ClusterStateMutator.createCollection building a new cName: collection1
   [junit4]   2> 100985 T453 oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard2
   [junit4]   2> 101087 T463 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 101087 T478 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 101088 T452 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 101984 T479 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 101984 T479 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 101985 T479 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 101985 T479 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 101986 T479 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 101986 T479 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-003/collection1/'
   [junit4]   2> 101987 T479 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud%20311931BA026040D2-001/tempDir-003/collection1/lib/.svn/' to classloader
   [junit4]   2> 101987 T479 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud%20311931BA026040D2-001/tempDir-003/collection1/lib/classes/' to classloader
   [junit4]   2> 101987 T479 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud%20311931BA026040D2-001/tempDir-003/collection1/lib/README' to classloader
   [junit4]   2> 102009 T479 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 102011 T479 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 102016 T479 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 102020 T479 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 102028 T479 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 102029 T479 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 102033 T479 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 102130 T479 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 102134 T479 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 102135 T479 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 102142 T479 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 102144 T479 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 102146 T479 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 102146 T479 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 102147 T479 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 102147 T479 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 102148 T479 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 102148 T479 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 102148 T479 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 102148 T479 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 102148 T479 oasc.SolrCore.<init> [collection1] Opening new SolrCore at /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-003/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-001/jetty1/
   [junit4]   2> 102149 T479 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@11deaa5
   [junit4]   2> 102149 T479 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-001/jetty1
   [junit4]   2> 102149 T479 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-001/jetty1/index/
   [junit4]   2> 102149 T479 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-001/jetty1/index' doesn't exist. Creating new index...
   [junit4]   2> 102150 T479 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-001/jetty1/index
   [junit4]   2> 102150 T479 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=21, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 102151 T479 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@1641c1d lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1c8bcc7),segFN=segments_1,generation=1}
   [junit4]   2> 102151 T479 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 102154 T479 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 102154 T479 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 102154 T479 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 102154 T479 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 102154 T479 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 102154 T479 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 102155 T479 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 102155 T479 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 102155 T479 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 102156 T479 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 102156 T479 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 102156 T479 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 102157 T479 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 102157 T479 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
   [junit4]   2> 102157 T479 oasc.RequestHandlers.initHandlersFromConfig created /schema: org.apache.solr.handler.SchemaHandler
   [junit4]   2> 102158 T479 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler
   [junit4]   2> 102158 T479 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler
   [junit4]   2> 102159 T479 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
   [junit4]   2> 102160 T479 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
   [junit4]   2> 102160 T479 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
   [junit4]   2> 102160 T479 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
   [junit4]   2> 102161 T479 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
   [junit4]   2> 102161 T479 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler
   [junit4]   2> 102161 T479 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
   [junit4]   2> 102162 T479 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler
   [junit4]   2> 102162 T479 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 102162 T479 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 102164 T479 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 102165 T479 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 102165 T479 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 102166 T479 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 102167 T479 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 102168 T479 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 102168 T479 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 102169 T479 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=32, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=67.88671875, floorSegmentMB=0.5439453125, forceMergeDeletesPctAllowed=2.3235970957651753, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 102169 T479 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@1641c1d lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@1c8bcc7),segFN=segments_1,generation=1}
   [junit4]   2> 102169 T479 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 102170 T479 oass.SolrIndexSearcher.<init> Opening Searcher@1d4e9d5[collection1] main
   [junit4]   2> 102170 T479 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 102171 T479 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 102171 T479 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 102171 T479 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 102171 T479 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 102172 T479 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 102172 T479 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 102173 T479 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 102173 T479 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 102173 T479 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 102174 T480 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@1d4e9d5[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 102174 T479 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 102175 T483 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:44431 collection:collection1 shard:shard2
   [junit4]   2> 102175 T426 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 102175 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard2/election
   [junit4]   2> 102176 T426 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 102178 T483 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard2
   [junit4]   2> 102179 T452 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 102180 T483 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 102180 T483 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C168 name=collection1 org.apache.solr.core.SolrCore@16ebaee url=http://127.0.0.1:44431/collection1 node=127.0.0.1:44431_ C168_STATE=coll:collection1 core:collection1 props:{core=collection1, base_url=http://127.0.0.1:44431, node_name=127.0.0.1:44431_, state=down}
   [junit4]   2> 102180 T483 C168 P44431 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:44431/collection1/
   [junit4]   2> 102180 T453 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1"} current state version: 4
   [junit4]   2> 102180 T483 C168 P44431 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 102181 T483 C168 P44431 oasc.SyncStrategy.syncToMe http://127.0.0.1:44431/collection1/ has no replicas
   [junit4]   2> 102181 T483 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:44431/collection1/ shard2
   [junit4]   2> 102181 T483 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard2
   [junit4]   2> 102185 T452 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 102186 T453 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:44431",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active"} current state version: 4
   [junit4]   2> 102288 T478 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 102289 T452 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 102288 T463 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 102336 T483 oasc.ZkController.register We are http://127.0.0.1:44431/collection1/ and leader is http://127.0.0.1:44431/collection1/
   [junit4]   2> 102336 T483 oasc.ZkController.register No LogReplay needed for core=collection1 baseURL=http://127.0.0.1:44431
   [junit4]   2> 102336 T483 oasc.ZkController.checkRecovery I am the leader, no recovery necessary
   [junit4]   2> 102336 T483 oasc.ZkController.publish publishing core=collection1 state=active collection=collection1
   [junit4]   2> 102336 T483 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 102337 T452 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 102338 T453 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:44431",
   [junit4]   2> 	  "node_name":"127.0.0.1:44431_",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"} current state version: 5
   [junit4]   2> 102338 T453 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:44431",
   [junit4]   2> 	  "node_name":"127.0.0.1:44431_",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"active",
   [junit4]   2> 	  "shard":"shard2",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 102441 T463 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 102441 T452 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 102441 T478 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 2)
   [junit4]   2> 102445 T426 oasc.AbstractFullDistribZkTestBase.createJettys create jetty 2
   [junit4]   2> 102446 T426 oejs.Server.doStart jetty-8.1.10.v20130312
   [junit4]   2> 102448 T426 oejs.AbstractConnector.doStart Started SelectChannelConnector@127.0.0.1:46867
   [junit4]   2> 102450 T426 oass.SolrDispatchFilter.init SolrDispatchFilter.init()sun.misc.Launcher$AppClassLoader@e2f2a
   [junit4]   2> 102450 T426 oasc.SolrResourceLoader.locateSolrHome JNDI not configured for solr (NoInitialContextEx)
   [junit4]   2> 102450 T426 oasc.SolrResourceLoader.locateSolrHome using system property solr.solr.home: ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-004
   [junit4]   2> 102451 T426 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-004/'
   [junit4]   2> 102462 T426 oasc.ConfigSolr.fromFile Loading container configuration from /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-004/solr.xml
   [junit4]   2> 102470 T426 oasc.CoreContainer.<init> New CoreContainer 13516631
   [junit4]   2> 102470 T426 oasc.CoreContainer.load Loading cores into CoreContainer [instanceDir=../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-004/]
   [junit4]   2> 102471 T426 oashc.HttpShardHandlerFactory.getParameter Setting socketTimeout to: 90000
   [junit4]   2> 102471 T426 oashc.HttpShardHandlerFactory.getParameter Setting urlScheme to: 
   [junit4]   2> 102471 T426 oashc.HttpShardHandlerFactory.getParameter Setting connTimeout to: 15000
   [junit4]   2> 102471 T426 oashc.HttpShardHandlerFactory.getParameter Setting maxConnectionsPerHost to: 20
   [junit4]   2> 102472 T426 oashc.HttpShardHandlerFactory.getParameter Setting maxConnections to: 10000
   [junit4]   2> 102472 T426 oashc.HttpShardHandlerFactory.getParameter Setting corePoolSize to: 0
   [junit4]   2> 102472 T426 oashc.HttpShardHandlerFactory.getParameter Setting maximumPoolSize to: 2147483647
   [junit4]   2> 102472 T426 oashc.HttpShardHandlerFactory.getParameter Setting maxThreadIdleTime to: 5
   [junit4]   2> 102473 T426 oashc.HttpShardHandlerFactory.getParameter Setting sizeOfQueue to: -1
   [junit4]   2> 102473 T426 oashc.HttpShardHandlerFactory.getParameter Setting fairnessPolicy to: false
   [junit4]   2> 102473 T426 oasu.UpdateShardHandler.<init> Creating UpdateShardHandler HTTP client with params: socketTimeout=340000&connTimeout=45000&retry=false
   [junit4]   2> 102473 T426 oasl.LogWatcher.createWatcher SLF4J impl is org.slf4j.impl.Log4jLoggerFactory
   [junit4]   2> 102474 T426 oasl.LogWatcher.newRegisteredLogWatcher Registering Log Listener [Log4j (org.slf4j.impl.Log4jLoggerFactory)]
   [junit4]   2> 102474 T426 oasc.CoreContainer.load Host Name: 127.0.0.1
   [junit4]   2> 102474 T426 oasc.ZkContainer.initZooKeeper Zookeeper client=127.0.0.1:47620/solr
   [junit4]   2> 102474 T426 oasc.ZkController.checkChrootPath zkHost includes chroot
   [junit4]   2> 102475 T426 oascc.SolrZkClient.createZkCredentialsToAddAutomatically Using default ZkCredentialsProvider
   [junit4]   2> 102475 T426 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 102477 T495 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@1c090d2 name:ZooKeeperConnection Watcher:127.0.0.1:47620 got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 102477 T426 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 102478 T426 oascc.SolrZkClient.createZkACLProvider Using default ZkACLProvider
   [junit4]   2> 102481 T426 oascc.ConnectionManager.waitForConnected Waiting for client to connect to ZooKeeper
   [junit4]   2> 102482 T498 oascc.ConnectionManager.process Watcher org.apache.solr.common.cloud.ConnectionManager@8cbf20 name:ZooKeeperConnection Watcher:127.0.0.1:47620/solr got event WatchedEvent state:SyncConnected type:None path:null path:null type:None
   [junit4]   2> 102483 T426 oascc.ConnectionManager.waitForConnected Client is connected to ZooKeeper
   [junit4]   2> 102486 T426 oascc.ZkStateReader.createClusterStateWatchersAndUpdate Updating cluster state from ZooKeeper... 
   [junit4]   2> 103489 T426 oasc.ZkController.createEphemeralLiveNode Register node as live in ZooKeeper:/live_nodes/127.0.0.1:46867_
   [junit4]   2> 103491 T426 oascc.SolrZkClient.makePath makePath: /live_nodes/127.0.0.1:46867_
   [junit4]   2> 103497 T426 oasc.Overseer.close Overseer (id=null) closing
   [junit4]   2> 103502 T499 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 103503 T499 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 103504 T452 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 103504 T499 oasc.ZkController.waitForCoreNodeName look for our core node name
   [junit4]   2> 103506 T453 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:46867",
   [junit4]   2> 	  "node_name":"127.0.0.1:46867_",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"} current state version: 6
   [junit4]   2> 103507 T453 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:46867",
   [junit4]   2> 	  "node_name":"127.0.0.1:46867_",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":null,
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 103507 T453 oasco.ReplicaMutator.updateState Collection already exists with numShards=2
   [junit4]   2> 103507 T453 oasco.ReplicaMutator.updateState Assigning new node to shard shard=shard1
   [junit4]   2> 103612 T463 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 103612 T498 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 103612 T452 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 103612 T478 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 104505 T499 oasc.ZkController.waitForShardId waiting to find shard id in clusterstate for collection1
   [junit4]   2> 104505 T499 oasc.ZkController.createCollectionZkNode Check for collection zkNode:collection1
   [junit4]   2> 104505 T499 oasc.ZkController.createCollectionZkNode Collection zkNode exists
   [junit4]   2> 104506 T499 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 104506 T499 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 104506 T499 oasc.SolrResourceLoader.<init> new SolrResourceLoader for directory: '../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-004/collection1/'
   [junit4]   2> 104507 T499 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud%20311931BA026040D2-001/tempDir-004/collection1/lib/.svn/' to classloader
   [junit4]   2> 104507 T499 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud%20311931BA026040D2-001/tempDir-004/collection1/lib/classes/' to classloader
   [junit4]   2> 104508 T499 oasc.SolrResourceLoader.replaceClassLoader Adding 'file:/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud%20311931BA026040D2-001/tempDir-004/collection1/lib/README' to classloader
   [junit4]   2> 104522 T499 oasc.ZkController.watchZKConfDir watch zkdir /configs/conf1
   [junit4]   2> 104525 T499 oasc.Config.<init> loaded config solrconfig.xml with version 0 
   [junit4]   2> 104531 T499 oasc.SolrConfig.refreshRequestParams current version of requestparams : -1
   [junit4]   2> 104536 T499 oasc.SolrConfig.<init> Using Lucene MatchVersion: 5.0.0
   [junit4]   2> 104544 T499 oasc.SolrConfig.<init> Loaded SolrConfig: solrconfig.xml
   [junit4]   2> 104546 T499 oass.IndexSchema.readSchema Reading Solr Schema from /configs/conf1/schema.xml
   [junit4]   2> 104561 T499 oass.IndexSchema.readSchema [collection1] Schema name=test
   [junit4]   2> 104652 T499 oass.OpenExchangeRatesOrgProvider.init Initialized with rates=open-exchange-rates.json, refreshInterval=1440.
   [junit4]   2> 104656 T499 oass.IndexSchema.readSchema default search field in schema is text
   [junit4]   2> 104657 T499 oass.IndexSchema.readSchema unique key field: id
   [junit4]   2> 104672 T499 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 104675 T499 oass.FileExchangeRateProvider.reload Reloading exchange rates from file currency.xml
   [junit4]   2> 104677 T499 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 104678 T499 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 104680 T499 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 104680 T499 oass.OpenExchangeRatesOrgProvider.reload Reloading exchange rates from open-exchange-rates.json
   [junit4]   2> 104681 T499 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Unknown key IMPORTANT NOTE
   [junit4]   2> 104681 T499 oass.OpenExchangeRatesOrgProvider$OpenExchangeRates.<init> WARN Expected key, got STRING
   [junit4]   2> 104682 T499 oasc.CoreContainer.create Creating SolrCore 'collection1' using configuration from collection collection1
   [junit4]   2> 104682 T499 oasc.SolrCore.initDirectoryFactory org.apache.solr.core.MockDirectoryFactory
   [junit4]   2> 104683 T499 oasc.SolrCore.<init> [collection1] Opening new SolrCore at ../../../../../../../../../../mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-004/collection1/, dataDir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-001/jetty2/
   [junit4]   2> 104683 T499 oasc.JmxMonitoredMap.<init> JMX monitoring is enabled. Adding Solr mbeans to JMX Server: com.sun.jmx.mbeanserver.JmxMBeanServer@11deaa5
   [junit4]   2> 104684 T499 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-001/jetty2
   [junit4]   2> 104685 T499 oasc.SolrCore.getNewIndexDir New index directory detected: old=null new=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-001/jetty2/index/
   [junit4]   2> 104685 T499 oasc.SolrCore.initIndex WARN [collection1] Solr index directory '/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-001/jetty2/index' doesn't exist. Creating new index...
   [junit4]   2> 104685 T499 oasc.CachingDirectoryFactory.get return new directory for /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-001/jetty2/index
   [junit4]   2> 104686 T499 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.LogByteSizeMergePolicy: [LogByteSizeMergePolicy: minMergeSize=1677721, mergeFactor=21, maxMergeSize=2147483648, maxMergeSizeForForcedMerge=9223372036854775807, calibrateSizeByDeletes=true, maxMergeDocs=2147483647, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=0.0]
   [junit4]   2> 104688 T499 oasc.SolrDeletionPolicy.onCommit SolrDeletionPolicy.onCommit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@1dbea08 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@9ff8b8),segFN=segments_1,generation=1}
   [junit4]   2> 104689 T499 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 104693 T499 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "nodistrib"
   [junit4]   2> 104693 T499 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "dedupe"
   [junit4]   2> 104694 T499 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "dedupe"
   [junit4]   2> 104694 T499 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "stored_sig"
   [junit4]   2> 104694 T499 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "stored_sig"
   [junit4]   2> 104695 T499 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-explicit"
   [junit4]   2> 104695 T499 oasup.UpdateRequestProcessorChain.init creating updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 104695 T499 oasup.UpdateRequestProcessorChain.init inserting DistributedUpdateProcessorFactory into updateRequestProcessorChain "distrib-dup-test-chain-implicit"
   [junit4]   2> 104696 T499 oasc.SolrCore.loadUpdateProcessorChains no updateRequestProcessorChain defined as default, creating implicit default
   [junit4]   2> 104696 T499 oasc.RequestHandlers.initHandlersFromConfig created /update: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 104696 T499 oasc.RequestHandlers.initHandlersFromConfig created /update/json: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 104697 T499 oasc.RequestHandlers.initHandlersFromConfig created /update/csv: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 104697 T499 oasc.RequestHandlers.initHandlersFromConfig created /update/json/docs: org.apache.solr.handler.UpdateRequestHandler
   [junit4]   2> 104697 T499 oasc.RequestHandlers.initHandlersFromConfig created /config: org.apache.solr.handler.SolrConfigHandler
   [junit4]   2> 104698 T499 oasc.RequestHandlers.initHandlersFromConfig created /schema: org.apache.solr.handler.SchemaHandler
   [junit4]   2> 104698 T499 oasc.RequestHandlers.initHandlersFromConfig created /replication: org.apache.solr.handler.ReplicationHandler
   [junit4]   2> 104699 T499 oasc.RequestHandlers.initHandlersFromConfig created /get: org.apache.solr.handler.RealTimeGetHandler
   [junit4]   2> 104699 T499 oasc.RequestHandlers.initHandlersFromConfig created /admin/luke: org.apache.solr.handler.admin.LukeRequestHandler
   [junit4]   2> 104700 T499 oasc.RequestHandlers.initHandlersFromConfig created /admin/system: org.apache.solr.handler.admin.SystemInfoHandler
   [junit4]   2> 104700 T499 oasc.RequestHandlers.initHandlersFromConfig created /admin/mbeans: org.apache.solr.handler.admin.SolrInfoMBeanHandler
   [junit4]   2> 104700 T499 oasc.RequestHandlers.initHandlersFromConfig created /admin/plugins: org.apache.solr.handler.admin.PluginInfoHandler
   [junit4]   2> 104701 T499 oasc.RequestHandlers.initHandlersFromConfig created /admin/threads: org.apache.solr.handler.admin.ThreadDumpHandler
   [junit4]   2> 104701 T499 oasc.RequestHandlers.initHandlersFromConfig created /admin/properties: org.apache.solr.handler.admin.PropertiesRequestHandler
   [junit4]   2> 104702 T499 oasc.RequestHandlers.initHandlersFromConfig created /admin/logging: org.apache.solr.handler.admin.LoggingHandler
   [junit4]   2> 104702 T499 oasc.RequestHandlers.initHandlersFromConfig created /admin/ping: org.apache.solr.handler.PingRequestHandler
   [junit4]   2> 104702 T499 oasc.RequestHandlers.initHandlersFromConfig created standard: solr.StandardRequestHandler
   [junit4]   2> 104703 T499 oasc.RequestHandlers.initHandlersFromConfig created /admin/file: solr.admin.ShowFileRequestHandler
   [junit4]   2> 104704 T499 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 104705 T499 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 104706 T499 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 104707 T499 oashl.XMLLoader.init xsltCacheLifetimeSeconds=60
   [junit4]   2> 104707 T499 oasc.SolrCore.initStatsCache Using default statsCache cache: org.apache.solr.search.stats.LocalStatsCache
   [junit4]   2> 104708 T499 oasu.CommitTracker.<init> Hard AutoCommit: disabled
   [junit4]   2> 104708 T499 oasu.CommitTracker.<init> Soft AutoCommit: disabled
   [junit4]   2> 104709 T499 oasu.RandomMergePolicy.<init> RandomMergePolicy wrapping class org.apache.lucene.index.TieredMergePolicy: [TieredMergePolicy: maxMergeAtOnce=32, maxMergeAtOnceExplicit=39, maxMergedSegmentMB=67.88671875, floorSegmentMB=0.5439453125, forceMergeDeletesPctAllowed=2.3235970957651753, segmentsPerTier=20.0, maxCFSSegmentSizeMB=8.796093022207999E12, noCFSRatio=1.0
   [junit4]   2> 104709 T499 oasc.SolrDeletionPolicy.onInit SolrDeletionPolicy.onInit: commits: num=1
   [junit4]   2> 		commit{dir=MockDirectoryWrapper(RAMDirectory@1dbea08 lockFactory=org.apache.lucene.store.SingleInstanceLockFactory@9ff8b8),segFN=segments_1,generation=1}
   [junit4]   2> 104710 T499 oasc.SolrDeletionPolicy.updateCommits newest commit generation = 1
   [junit4]   2> 104710 T499 oass.SolrIndexSearcher.<init> Opening Searcher@9ea2ea[collection1] main
   [junit4]   2> 104710 T499 oascc.ZkStateReader.readConfigName Load collection config from:/collections/collection1
   [junit4]   2> 104712 T499 oascc.ZkStateReader.readConfigName path=/collections/collection1 configName=conf1 specified config exists in ZooKeeper
   [junit4]   2> 104712 T499 oasr.ManagedResourceStorage.newStorageIO Setting up ZooKeeper-based storage for the RestManager with znodeBase: /configs/conf1
   [junit4]   2> 104712 T499 oasr.ManagedResourceStorage$ZooKeeperStorageIO.configure Configured ZooKeeperStorageIO with znodeBase: /configs/conf1
   [junit4]   2> 104713 T499 oasr.RestManager.init Initializing RestManager with initArgs: {}
   [junit4]   2> 104713 T499 oasr.ManagedResourceStorage.load Reading _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 104713 T499 oasr.ManagedResourceStorage$ZooKeeperStorageIO.openInputStream No data found for znode /configs/conf1/_rest_managed.json
   [junit4]   2> 104714 T499 oasr.ManagedResourceStorage.load Loaded null at path _rest_managed.json using ZooKeeperStorageIO:path=/configs/conf1
   [junit4]   2> 104714 T499 oasr.RestManager.init Initializing 0 registered ManagedResources
   [junit4]   2> 104714 T499 oash.ReplicationHandler.inform Commits will be reserved for  10000
   [junit4]   2> 104715 T500 oasc.SolrCore.registerSearcher [collection1] Registered new searcher Searcher@9ea2ea[collection1] main{ExitableDirectoryReader(UninvertingDirectoryReader())}
   [junit4]   2> 104715 T499 oasc.CoreContainer.registerCore registering core: collection1
   [junit4]   2> 104715 T503 oasc.ZkController.register Register replica - core:collection1 address:http://127.0.0.1:46867 collection:collection1 shard:shard1
   [junit4]   2> 104715 T426 oass.SolrDispatchFilter.init user.dir=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0
   [junit4]   2> 104716 T426 oass.SolrDispatchFilter.init SolrDispatchFilter.init() done
   [junit4]   2> 104716 T503 oascc.SolrZkClient.makePath makePath: /collections/collection1/leader_elect/shard1/election
   [junit4]   2> 104719 T503 oasc.ShardLeaderElectionContext.runLeaderProcess Running the leader process for shard shard1
   [junit4]   2> 104720 T452 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 104720 T503 oasc.ShardLeaderElectionContext.waitForReplicasToComeUp Enough replicas found to continue.
   [junit4]   2> 104720 T503 oasc.ShardLeaderElectionContext.runLeaderProcess I may be the new leader - try and sync
   [junit4]   2> ASYNC  NEW_CORE C169 name=collection1 org.apache.solr.core.SolrCore@d37174 url=http://127.0.0.1:46867/collection1 node=127.0.0.1:46867_ C169_STATE=coll:collection1 core:collection1 props:{core=collection1, base_url=http://127.0.0.1:46867, node_name=127.0.0.1:46867_, state=down}
   [junit4]   2> 104721 T503 C169 P46867 oasc.SyncStrategy.sync Sync replicas to http://127.0.0.1:46867/collection1/
   [junit4]   2> 104721 T453 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1"} current state version: 7
   [junit4]   2> 104721 T503 C169 P46867 oasc.SyncStrategy.syncReplicas Sync Success - now sync replicas to me
   [junit4]   2> 104721 T503 C169 P46867 oasc.SyncStrategy.syncToMe http://127.0.0.1:46867/collection1/ has no replicas
   [junit4]   2> 104721 T503 oasc.ShardLeaderElectionContext.runLeaderProcess I am the new leader: http://127.0.0.1:46867/collection1/ shard1
   [junit4]   2> 104721 T503 oascc.SolrZkClient.makePath makePath: /collections/collection1/leaders/shard1
   [junit4]   2> 104724 T452 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 104725 T453 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "operation":"leader",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "base_url":"http://127.0.0.1:46867",
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "state":"active"} current state version: 7
   [junit4]   2> 104827 T478 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 104827 T452 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 104827 T463 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 104827 T498 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 3)
   [junit4]   2> 104875 T503 oasc.ZkCon

[...truncated too long message...]

 current state version: 22
   [junit4]   2> 131604 T540 oascc.ZkStateReader$2.process A cluster state change: WatchedEvent state:SyncConnected type:NodeDataChanged path:/clusterstate.json, has occurred - updating... (live nodes size: 1)
   [junit4]   2> 132334 T518 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 132359 T426 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
   [junit4]   2> 132547 T426 oasc.ChaosMonkey.monkeyLog monkey: stop shard! 41246
   [junit4]   2> 132548 T426 oasc.CoreContainer.shutdown Shutting down CoreContainer instance=3796594
   [junit4]   2> 132548 T426 oasc.ZkController.publish publishing core=collection1 state=down collection=collection1
   [junit4]   2> 132549 T426 oasc.ZkController.publish numShards not found on descriptor - reading it from system property
   [junit4]   2> 132551 T540 oasc.DistributedQueue$LatchWatcher.process NodeChildrenChanged fired on path /overseer/queue state SyncConnected
   [junit4]   2> 132551 T426 oasc.SolrCore.close [collection1]  CLOSING SolrCore org.apache.solr.core.SolrCore@16e5cbc
   [junit4]   2> 132552 T426 oasc.ZkController.unRegisterConfListener  a listener was removed because of core close
   [junit4]   2> 132552 T426 oasu.DirectUpdateHandler2.close closing DirectUpdateHandler2{commits=0,autocommits=0,soft autocommits=0,optimizes=0,rollbacks=0,expungeDeletes=0,docsPending=0,adds=0,deletesById=0,deletesByQuery=0,errors=0,cumulative_adds=0,cumulative_deletesById=0,cumulative_deletesByQuery=0,cumulative_errors=0,transaction_logs_total_size=0,transaction_logs_total_number=0}
   [junit4]   2> 132553 T638 oasc.Overseer$ClusterStateUpdater.run processMessage: queueSize: 1, message = {
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node4",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:41246",
   [junit4]   2> 	  "node_name":"127.0.0.1:41246_",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"} current state version: 23
   [junit4]   2> 132553 T426 oasu.SolrCoreState.decrefSolrCoreState Closing SolrCoreState
   [junit4]   2> 132554 T426 oasu.DefaultSolrCoreState.closeIndexWriter SolrCoreState ref count has reached 0 - closing IndexWriter
   [junit4]   2> 132554 T426 oasu.DefaultSolrCoreState.closeIndexWriter closing IndexWriter with IndexWriterCloser
   [junit4]   2> 132554 T638 oasco.ReplicaMutator.updateState Update state numShards=2 message={
   [junit4]   2> 	  "core":"collection1",
   [junit4]   2> 	  "core_node_name":"core_node4",
   [junit4]   2> 	  "roles":null,
   [junit4]   2> 	  "base_url":"http://127.0.0.1:41246",
   [junit4]   2> 	  "node_name":"127.0.0.1:41246_",
   [junit4]   2> 	  "numShards":"2",
   [junit4]   2> 	  "state":"down",
   [junit4]   2> 	  "shard":"shard1",
   [junit4]   2> 	  "collection":"collection1",
   [junit4]   2> 	  "operation":"state"}
   [junit4]   2> 132556 T426 oasc.SolrCore.closeSearcher [collection1] Closing main searcher on request.
   [junit4]   2> 132565 T426 oasc.CachingDirectoryFactory.close Closing MockDirectoryFactory - 2 directories currently being tracked
   [junit4]   2> 132565 T426 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-001/jetty4/index [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-001/jetty4/index;done=false>>]
   [junit4]   2> 132566 T426 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-001/jetty4/index
   [junit4]   2> 132567 T426 oasc.CachingDirectoryFactory.closeCacheValue looking to close /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-001/jetty4 [CachedDir<<refCount=0;path=/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-001/jetty4;done=false>>]
   [junit4]   2> 132567 T426 oasc.CachingDirectoryFactory.close Closing directory: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001/tempDir-001/jetty4
   [junit4]   2> 132569 T426 oasc.Overseer.close Overseer (id=93109939143245836-127.0.0.1:41246_-n_0000000004) closing
   [junit4]   2> 132569 T638 oasc.Overseer$ClusterStateUpdater.run Overseer Loop exiting : 127.0.0.1:41246_
   [junit4]   2> 132573 T540 oascc.ZkStateReader$3.process WARN ZooKeeper watch triggered, but Solr cannot talk to ZK
   [junit4]   2> 132597 T426 oejsh.ContextHandler.doStop stopped o.e.j.s.ServletContextHandler{/,null}
   [junit4]   2> 132793 T426 oas.SolrTestCaseJ4.tearDown ###Ending testDistribSearch
   [junit4]   2> 132794 T426 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:47620 47620
   [junit4]   2> 133338 T427 oasc.ZkTestServer.send4LetterWord connecting to 127.0.0.1:47620 47620
   [junit4]   2> 133341 T427 oasc.ZkTestServer$ZKServerMain.runFromConfig WARN Watch limit violations: 
   [junit4]   2> 	Maximum concurrent create/delete watches above limit:
   [junit4]   2> 	
   [junit4]   2> 		45	/solr/configs/conf1
   [junit4]   2> 		6	/solr/aliases.json
   [junit4]   2> 		6	/solr/clusterstate.json
   [junit4]   2> 	
   [junit4]   2> 	Maximum concurrent children watches above limit:
   [junit4]   2> 	
   [junit4]   2> 		6	/solr/live_nodes
   [junit4]   2> 		5	/solr/overseer/queue
   [junit4]   2> 		5	/solr/overseer/collection-queue-work
   [junit4]   2> 	
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestSolrConfigHandlerCloud -Dtests.method=testDistribSearch -Dtests.seed=311931BA026040D2 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=el -Dtests.timezone=Africa/Porto-Novo -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 35.4s J0 | TestSolrConfigHandlerCloud.testDistribSearch <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: Could not get expected value  CY val modified for path [response, params, y, c] full output {
   [junit4]    >   "responseHeader":{
   [junit4]    >     "status":0,
   [junit4]    >     "QTime":0},
   [junit4]    >   "response":{
   [junit4]    >     "znodeVersion":0,
   [junit4]    >     "params":{"x":{
   [junit4]    >         "a":"A val",
   [junit4]    >         "b":"B val",
   [junit4]    >         "":{"v":0}}}}}
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([311931BA026040D2:B0FFBFA2753F20EE]:0)
   [junit4]    > 	at org.apache.solr.core.TestSolrConfigHandler.testForResponseElement(TestSolrConfigHandler.java:256)
   [junit4]    > 	at org.apache.solr.handler.TestSolrConfigHandlerCloud.testReqParams(TestSolrConfigHandlerCloud.java:225)
   [junit4]    > 	at org.apache.solr.handler.TestSolrConfigHandlerCloud.doTest(TestSolrConfigHandlerCloud.java:61)
   [junit4]    > 	at org.apache.solr.BaseDistributedSearchTestCase.testDistribSearch(BaseDistributedSearchTestCase.java:868)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> 133380 T426 oas.SolrTestCaseJ4.deleteCore ###deleteCore
   [junit4]   2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build/solr-core/test/J0/temp/solr.handler.TestSolrConfigHandlerCloud 311931BA026040D2-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50), sim=DefaultSimilarity, locale=el, timezone=Africa/Porto-Novo
   [junit4]   2> NOTE: Linux 3.13.0-43-generic i386/Oracle Corporation 1.8.0_40-ea (32-bit)/cpus=8,threads=1,free=78360392,total=215228416
   [junit4]   2> NOTE: All tests run in this JVM: [ZkControllerTest, PreAnalyzedFieldTest, DistributedFacetPivotLongTailTest, PingRequestHandlerTest, TriLevelCompositeIdRoutingTest, TestLeaderElectionZkExpiry, RequestHandlersTest, TestCSVLoader, ExitableDirectoryReaderTest, IndexSchemaRuntimeFieldTest, TestStressLucene, HardAutoCommitTest, IgnoreCommitOptimizeUpdateProcessorFactoryTest, TestSimpleQParserPlugin, TestSolrConfigHandlerCloud]
   [junit4] Completed on J0 in 35.43s, 1 test, 1 failure <<< FAILURES!

[...truncated 1373 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:529: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:477: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:61: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/extra-targets.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/build.xml:190: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/solr/common-build.xml:508: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:1363: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:970: There were test failures: 459 suites, 1867 tests, 1 failure, 47 ignored (22 assumptions)

Total time: 70 minutes 2 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 32bit/jdk1.8.0_40-ea-b20 -server -XX:+UseSerialGC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any



[JENKINS] Lucene-Solr-5.x-Linux (32bit/jdk1.9.0-ea-b44) - Build # 11382 - Still Failing!

Posted by Policeman Jenkins Server <je...@thetaphi.de>.
Build: http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/11382/
Java: 32bit/jdk1.9.0-ea-b44 -client -XX:+UseG1GC

34 tests failed.
FAILED:  junit.framework.TestSuite.org.apache.lucene.search.TestControlledRealTimeReopenThread

Error Message:
13 threads leaked from SUITE scope at org.apache.lucene.search.TestControlledRealTimeReopenThread:     1) Thread[id=854, name=TestControlledRealTimeReopenThread-3-thread-6, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=850, name=TestControlledRealTimeReopenThread-3-thread-2, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=844, name=NRTDeletes Reopen Thread, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.apache.lucene.search.ControlledRealTimeReopenThread.run(ControlledRealTimeReopenThread.java:223)    4) Thread[id=856, name=TestControlledRealTimeReopenThread-3-thread-8, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=857, name=TestControlledRealTimeReopenThread-3-thread-9, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=855, name=TestControlledRealTimeReopenThread-3-thread-7, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=859, name=TestControlledRealTimeReopenThread-3-thread-11, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=858, name=TestControlledRealTimeReopenThread-3-thread-10, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=845, name=NRTNoDeletes Reopen Thread, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)         at org.apache.lucene.search.ControlledRealTimeReopenThread.run(ControlledRealTimeReopenThread.java:223)   10) Thread[id=849, name=TestControlledRealTimeReopenThread-3-thread-1, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   11) Thread[id=851, name=TestControlledRealTimeReopenThread-3-thread-3, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   12) Thread[id=852, name=TestControlledRealTimeReopenThread-3-thread-4, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   13) Thread[id=853, name=TestControlledRealTimeReopenThread-3-thread-5, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: 13 threads leaked from SUITE scope at org.apache.lucene.search.TestControlledRealTimeReopenThread: 
   1) Thread[id=854, name=TestControlledRealTimeReopenThread-3-thread-6, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=850, name=TestControlledRealTimeReopenThread-3-thread-2, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=844, name=NRTDeletes Reopen Thread, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.apache.lucene.search.ControlledRealTimeReopenThread.run(ControlledRealTimeReopenThread.java:223)
   4) Thread[id=856, name=TestControlledRealTimeReopenThread-3-thread-8, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=857, name=TestControlledRealTimeReopenThread-3-thread-9, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=855, name=TestControlledRealTimeReopenThread-3-thread-7, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=859, name=TestControlledRealTimeReopenThread-3-thread-11, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=858, name=TestControlledRealTimeReopenThread-3-thread-10, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   9) Thread[id=845, name=NRTNoDeletes Reopen Thread, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
        at org.apache.lucene.search.ControlledRealTimeReopenThread.run(ControlledRealTimeReopenThread.java:223)
  10) Thread[id=849, name=TestControlledRealTimeReopenThread-3-thread-1, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
  11) Thread[id=851, name=TestControlledRealTimeReopenThread-3-thread-3, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
  12) Thread[id=852, name=TestControlledRealTimeReopenThread-3-thread-4, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
  13) Thread[id=853, name=TestControlledRealTimeReopenThread-3-thread-5, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([2BE3C7E61CEFFEA1]:0)


FAILED:  junit.framework.TestSuite.org.apache.lucene.search.TestControlledRealTimeReopenThread

Error Message:
There are still zombie threads that couldn't be terminated:    1) Thread[id=854, name=TestControlledRealTimeReopenThread-3-thread-6, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    2) Thread[id=850, name=TestControlledRealTimeReopenThread-3-thread-2, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    3) Thread[id=856, name=TestControlledRealTimeReopenThread-3-thread-8, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    4) Thread[id=857, name=TestControlledRealTimeReopenThread-3-thread-9, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    5) Thread[id=855, name=TestControlledRealTimeReopenThread-3-thread-7, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    6) Thread[id=859, name=TestControlledRealTimeReopenThread-3-thread-11, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    7) Thread[id=858, name=TestControlledRealTimeReopenThread-3-thread-10, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    8) Thread[id=849, name=TestControlledRealTimeReopenThread-3-thread-1, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)    9) Thread[id=851, name=TestControlledRealTimeReopenThread-3-thread-3, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   10) Thread[id=852, name=TestControlledRealTimeReopenThread-3-thread-4, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)   11) Thread[id=853, name=TestControlledRealTimeReopenThread-3-thread-5, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]         at sun.misc.Unsafe.park(Native Method)         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)         at java.lang.Thread.run(Thread.java:745)

Stack Trace:
com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:
   1) Thread[id=854, name=TestControlledRealTimeReopenThread-3-thread-6, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   2) Thread[id=850, name=TestControlledRealTimeReopenThread-3-thread-2, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   3) Thread[id=856, name=TestControlledRealTimeReopenThread-3-thread-8, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   4) Thread[id=857, name=TestControlledRealTimeReopenThread-3-thread-9, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   5) Thread[id=855, name=TestControlledRealTimeReopenThread-3-thread-7, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   6) Thread[id=859, name=TestControlledRealTimeReopenThread-3-thread-11, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   7) Thread[id=858, name=TestControlledRealTimeReopenThread-3-thread-10, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   8) Thread[id=849, name=TestControlledRealTimeReopenThread-3-thread-1, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
   9) Thread[id=851, name=TestControlledRealTimeReopenThread-3-thread-3, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
  10) Thread[id=852, name=TestControlledRealTimeReopenThread-3-thread-4, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
  11) Thread[id=853, name=TestControlledRealTimeReopenThread-3-thread-5, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
        at sun.misc.Unsafe.park(Native Method)
        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
        at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
        at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
        at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
        at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
	at __randomizedtesting.SeedInfo.seed([2BE3C7E61CEFFEA1]:0)


FAILED:  org.apache.lucene.codecs.perfield.TestPerFieldDocValuesFormat.testBinaryFixedLengthVsStoredFields

Error Message:
On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.codecs.perfield.TestPerFieldDocValuesFormat 2BE3C7E61CEFFEA1-001/index-MMapDirectory-001: Is a directory

Stack Trace:
java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.codecs.perfield.TestPerFieldDocValuesFormat 2BE3C7E61CEFFEA1-001/index-MMapDirectory-001: Is a directory
	at __randomizedtesting.SeedInfo.seed([2BE3C7E61CEFFEA1:865C6622006554CB]:0)
	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:418)
	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:236)
	at org.apache.lucene.store.MockDirectoryWrapper.renameFile(MockDirectoryWrapper.java:268)
	at org.apache.lucene.index.SegmentInfos.finishCommit(SegmentInfos.java:727)
	at org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:2914)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2895)
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2855)
	at org.apache.lucene.index.RandomIndexWriter.commit(RandomIndexWriter.java:257)
	at org.apache.lucene.index.BaseDocValuesFormatTestCase.doTestBinaryVsStoredFields(BaseDocValuesFormatTestCase.java:1363)
	at org.apache.lucene.index.BaseDocValuesFormatTestCase.testBinaryFixedLengthVsStoredFields(BaseDocValuesFormatTestCase.java:1408)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.lucene.index.TestIndexWriter.testHighFreqTerm

Error Message:
On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J0/temp/lucene.index.TestIndexWriter 2BE3C7E61CEFFEA1-001/index-MMapDirectory-001: Is a directory

Stack Trace:
java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J0/temp/lucene.index.TestIndexWriter 2BE3C7E61CEFFEA1-001/index-MMapDirectory-001: Is a directory
	at __randomizedtesting.SeedInfo.seed([2BE3C7E61CEFFEA1:4E17A3C224BE95BB]:0)
	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:418)
	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:236)
	at org.apache.lucene.store.MockDirectoryWrapper.renameFile(MockDirectoryWrapper.java:268)
	at org.apache.lucene.index.SegmentInfos.finishCommit(SegmentInfos.java:727)
	at org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:2914)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2895)
	at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:965)
	at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1008)
	at org.apache.lucene.index.TestIndexWriter.testHighFreqTerm(TestIndexWriter.java:497)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.lucene.index.TestOmitNorms.testMixedMerge

Error Message:
On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J0/temp/lucene.index.TestOmitNorms 2BE3C7E61CEFFEA1-001/index-MMapDirectory-001: Is a directory

Stack Trace:
java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J0/temp/lucene.index.TestOmitNorms 2BE3C7E61CEFFEA1-001/index-MMapDirectory-001: Is a directory
	at __randomizedtesting.SeedInfo.seed([2BE3C7E61CEFFEA1:411629C151DEEBB5]:0)
	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:418)
	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:236)
	at org.apache.lucene.store.MockDirectoryWrapper.renameFile(MockDirectoryWrapper.java:268)
	at org.apache.lucene.index.SegmentInfos.finishCommit(SegmentInfos.java:727)
	at org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:2914)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2895)
	at org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:965)
	at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1008)
	at org.apache.lucene.index.TestOmitNorms.testMixedMerge(TestOmitNorms.java:121)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.lucene.index.TestTermsEnum.testIntersectEmptyString

Error Message:
On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J0/temp/lucene.index.TestTermsEnum 2BE3C7E61CEFFEA1-001/index-MMapDirectory-001: Is a directory

Stack Trace:
java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J0/temp/lucene.index.TestTermsEnum 2BE3C7E61CEFFEA1-001/index-MMapDirectory-001: Is a directory
	at __randomizedtesting.SeedInfo.seed([2BE3C7E61CEFFEA1:FE6247D1288C218C]:0)
	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:418)
	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:236)
	at org.apache.lucene.store.MockDirectoryWrapper.renameFile(MockDirectoryWrapper.java:268)
	at org.apache.lucene.index.SegmentInfos.finishCommit(SegmentInfos.java:727)
	at org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:2914)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2895)
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2855)
	at org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:335)
	at org.apache.lucene.index.RandomIndexWriter.getReader(RandomIndexWriter.java:274)
	at org.apache.lucene.index.TestTermsEnum.testIntersectEmptyString(TestTermsEnum.java:843)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.lucene.codecs.perfield.TestPerFieldDocValuesFormat.testSortedSetVariableLengthSingleValuedVsStoredFields

Error Message:
On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.codecs.perfield.TestPerFieldDocValuesFormat 2BE3C7E61CEFFEA1-001/dvduel-001: Is a directory

Stack Trace:
java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.codecs.perfield.TestPerFieldDocValuesFormat 2BE3C7E61CEFFEA1-001/dvduel-001: Is a directory
	at __randomizedtesting.SeedInfo.seed([2BE3C7E61CEFFEA1:DB908CD036ECEE52]:0)
	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:418)
	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:236)
	at org.apache.lucene.store.MockDirectoryWrapper.renameFile(MockDirectoryWrapper.java:268)
	at org.apache.lucene.index.SegmentInfos.finishCommit(SegmentInfos.java:727)
	at org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:2914)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2895)
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2855)
	at org.apache.lucene.index.RandomIndexWriter.commit(RandomIndexWriter.java:257)
	at org.apache.lucene.index.BaseDocValuesFormatTestCase.doTestSortedSetVsStoredFields(BaseDocValuesFormatTestCase.java:1977)
	at org.apache.lucene.index.BaseDocValuesFormatTestCase.testSortedSetVariableLengthSingleValuedVsStoredFields(BaseDocValuesFormatTestCase.java:2129)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.lucene.codecs.perfield.TestPerFieldDocValuesFormat.testSortedSetVariableLengthVsStoredFields

Error Message:
On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.codecs.perfield.TestPerFieldDocValuesFormat 2BE3C7E61CEFFEA1-001/dvduel-002: Is a directory

Stack Trace:
java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.codecs.perfield.TestPerFieldDocValuesFormat 2BE3C7E61CEFFEA1-001/dvduel-002: Is a directory
	at __randomizedtesting.SeedInfo.seed([2BE3C7E61CEFFEA1:FF2961A7813E90C]:0)
	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:418)
	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:236)
	at org.apache.lucene.store.MockDirectoryWrapper.renameFile(MockDirectoryWrapper.java:268)
	at org.apache.lucene.index.SegmentInfos.finishCommit(SegmentInfos.java:727)
	at org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:2914)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2895)
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2855)
	at org.apache.lucene.index.RandomIndexWriter.commit(RandomIndexWriter.java:257)
	at org.apache.lucene.index.BaseDocValuesFormatTestCase.doTestSortedSetVsStoredFields(BaseDocValuesFormatTestCase.java:1977)
	at org.apache.lucene.index.BaseDocValuesFormatTestCase.testSortedSetVariableLengthVsStoredFields(BaseDocValuesFormatTestCase.java:2112)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.lucene.codecs.perfield.TestPerFieldDocValuesFormat.testSortedVariableLengthVsStoredFields

Error Message:
On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.codecs.perfield.TestPerFieldDocValuesFormat 2BE3C7E61CEFFEA1-001/dvduel-003: Is a directory

Stack Trace:
java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.codecs.perfield.TestPerFieldDocValuesFormat 2BE3C7E61CEFFEA1-001/dvduel-003: Is a directory
	at __randomizedtesting.SeedInfo.seed([2BE3C7E61CEFFEA1:18590A8C22DF08EA]:0)
	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:418)
	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:236)
	at org.apache.lucene.store.MockDirectoryWrapper.renameFile(MockDirectoryWrapper.java:268)
	at org.apache.lucene.index.SegmentInfos.finishCommit(SegmentInfos.java:727)
	at org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:2914)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2895)
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2855)
	at org.apache.lucene.index.RandomIndexWriter.commit(RandomIndexWriter.java:257)
	at org.apache.lucene.index.BaseDocValuesFormatTestCase.doTestSortedVsStoredFields(BaseDocValuesFormatTestCase.java:1446)
	at org.apache.lucene.index.BaseDocValuesFormatTestCase.testSortedVariableLengthVsStoredFields(BaseDocValuesFormatTestCase.java:1498)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.lucene.codecs.perfield.TestPerFieldDocValuesFormat.testSortedSetFixedLengthSingleValuedVsStoredFields

Error Message:
On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.codecs.perfield.TestPerFieldDocValuesFormat 2BE3C7E61CEFFEA1-001/dvduel-004: Is a directory

Stack Trace:
java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.codecs.perfield.TestPerFieldDocValuesFormat 2BE3C7E61CEFFEA1-001/dvduel-004: Is a directory
	at __randomizedtesting.SeedInfo.seed([2BE3C7E61CEFFEA1:AF927200FF9665BC]:0)
	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:418)
	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:236)
	at org.apache.lucene.store.MockDirectoryWrapper.renameFile(MockDirectoryWrapper.java:268)
	at org.apache.lucene.index.SegmentInfos.finishCommit(SegmentInfos.java:727)
	at org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:2914)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2895)
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2855)
	at org.apache.lucene.index.RandomIndexWriter.commit(RandomIndexWriter.java:257)
	at org.apache.lucene.index.BaseDocValuesFormatTestCase.doTestSortedSetVsStoredFields(BaseDocValuesFormatTestCase.java:1977)
	at org.apache.lucene.index.BaseDocValuesFormatTestCase.testSortedSetFixedLengthSingleValuedVsStoredFields(BaseDocValuesFormatTestCase.java:2121)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.lucene.codecs.perfield.TestPerFieldDocValuesFormat.testSortedSetFixedLengthVsStoredFields

Error Message:
On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.codecs.perfield.TestPerFieldDocValuesFormat 2BE3C7E61CEFFEA1-001/dvduel-005: Is a directory

Stack Trace:
java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.codecs.perfield.TestPerFieldDocValuesFormat 2BE3C7E61CEFFEA1-001/dvduel-005: Is a directory
	at __randomizedtesting.SeedInfo.seed([2BE3C7E61CEFFEA1:5E90762A0B12A6F7]:0)
	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:418)
	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:236)
	at org.apache.lucene.store.MockDirectoryWrapper.renameFile(MockDirectoryWrapper.java:268)
	at org.apache.lucene.index.SegmentInfos.finishCommit(SegmentInfos.java:727)
	at org.apache.lucene.index.IndexWriter.finishCommit(IndexWriter.java:2914)
	at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2895)
	at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2855)
	at org.apache.lucene.index.RandomIndexWriter.commit(RandomIndexWriter.java:257)
	at org.apache.lucene.index.BaseDocValuesFormatTestCase.doTestSortedSetVsStoredFields(BaseDocValuesFormatTestCase.java:1977)
	at org.apache.lucene.index.BaseDocValuesFormatTestCase.testSortedSetFixedLengthVsStoredFields(BaseDocValuesFormatTestCase.java:2041)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1618)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:827)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:863)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:877)
	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:836)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:738)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:772)
	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:783)
	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
	at com.carrotsearch.randomizedtesting.rules.SystemPropertiesInvariantRule$1.evaluate(SystemPropertiesInvariantRule.java:55)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
	at java.lang.Thread.run(Thread.java:745)


FAILED:  org.apache.lucene.codecs.perfield.TestPerFieldDocValuesFormat.testThreads3

Error Message:
On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.codecs.perfield.TestPerFieldDocValuesFormat 2BE3C7E61CEFFEA1-001/tempDir-001: Is a directory

Stack Trace:
java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemExcept

[...truncated too long message...]

hreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    8) Thread[id=849, name=TestControlledRealTimeReopenThread-3-thread-1, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >    9) Thread[id=851, name=TestControlledRealTimeReopenThread-3-thread-3, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   10) Thread[id=852, name=TestControlledRealTimeReopenThread-3-thread-4, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    >   11) Thread[id=853, name=TestControlledRealTimeReopenThread-3-thread-5, state=TIMED_WAITING, group=TGRP-TestControlledRealTimeReopenThread]
   [junit4]    >         at sun.misc.Unsafe.park(Native Method)
   [junit4]    >         at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
   [junit4]    >         at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
   [junit4]    >         at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
   [junit4]    >         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
   [junit4]    >         at java.lang.Thread.run(Thread.java:745)
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([2BE3C7E61CEFFEA1]:0)
   [junit4] Completed on J0 in 26.83s, 5 tests, 1 failure, 2 errors, 1 skipped <<< FAILURES!

[...truncated 645 lines...]
   [junit4] Suite: org.apache.lucene.store.TestMmapDirectory
   [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestMmapDirectory -Dtests.method=testRename -Dtests.seed=2BE3C7E61CEFFEA1 -Dtests.multiplier=3 -Dtests.slow=true -Dtests.locale=hi_IN -Dtests.timezone=America/Pangnirtung -Dtests.asserts=true -Dtests.file.encoding=UTF-8
   [junit4] FAILURE 0.01s J1 | TestMmapDirectory.testRename <<<
   [junit4]    > Throwable #1: java.lang.AssertionError: On Linux and MacOSX fsyncing a directory should not throw IOException, we just don't want to rely on that in production (undocumented). Got: java.nio.file.FileSystemException: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.store.TestMmapDirectory 2BE3C7E61CEFFEA1-001/testRename-001: Is a directory
   [junit4]    > 	at __randomizedtesting.SeedInfo.seed([2BE3C7E61CEFFEA1:87876148C99023F8]:0)
   [junit4]    > 	at org.apache.lucene.util.IOUtils.fsync(IOUtils.java:418)
   [junit4]    > 	at org.apache.lucene.store.FSDirectory.renameFile(FSDirectory.java:236)
   [junit4]    > 	at org.apache.lucene.store.BaseDirectoryTestCase.testRename(BaseDirectoryTestCase.java:109)
   [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
   [junit4]   2> NOTE: leaving temporary files on disk at: /mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build/core/test/J1/temp/lucene.store.TestMmapDirectory 2BE3C7E61CEFFEA1-001
   [junit4]   2> NOTE: test params are: codec=Asserting(Lucene50): {}, docValues:{}, sim=RandomSimilarityProvider(queryNorm=true,coord=yes): {}, locale=hi_IN, timezone=America/Pangnirtung
   [junit4]   2> NOTE: Linux 3.13.0-43-generic i386/Oracle Corporation 1.9.0-ea (32-bit)/cpus=8,threads=1,free=220001384,total=443547648
   [junit4]   2> NOTE: All tests run in this JVM: [TestMockFilesystems, TestDoc, TestLongPostings, TestIndexWriterMerging, TestBagOfPositions, TestTerm, TestSortedSetSortField, TestCloseableThreadLocal, TestPerFieldDocValuesFormat, TestSortRescorer, TestMinShouldMatch2, TestBooleanCoord, TestMixedCodecs, TestSearchAfter, TestSegmentTermDocs, TestBinaryDocument, TestFlushByRamOrCountsPolicy, TestEarlyTermination, TestManyFields, TestNGramPhraseQuery, TestDisjunctionMaxQuery, TestIndexWriterConfig, TestNewestSegment, TestMockDirectoryWrapper, TestRegexpQuery, TestMockAnalyzer, TestBinaryDocValuesUpdates, TestOmitPositions, TestTermRangeFilter, TestQueryRescorer, TestTermRangeQuery, TestPayloadNearQuery, TestSimpleAttributeImpl, TestQueryWrapperFilter, TestDirectPacked, TestCheckIndex, TestMultiTermQueryRewrites, TestComplexExplanationsOfNonMatches, TestLRUFilterCache, TestBooleanMinShouldMatch, TestSparseFixedBitDocIdSet, TestNearSpansOrdered, TestMinimize, TestCrash, TestIndexWriterOutOfMemory, TestHighCompressionMode, TestSameTokenSamePosition, TestAssertingTermVectorsFormat, TestDocIdSet, TestTopDocsMerge, TestReusableStringReader, TestFastCompressionMode, TestDocValuesScoring, TestFieldCacheRewriteMethod, TestRegexpRandom, TestSortedNumericSortField, TestNIOFSDirectory, TestSpans, TestDocTermOrdsRewriteMethod, TestIndexWriterUnicode, TestNumericRangeQuery64, TestMultiTermConstantScore, TestBitDocIdSetBuilder, TestArrayUtil, TestLevenshteinAutomata, TestMultiMMap, TestTopFieldCollector, TestByteBlockPool, TestFilterIterator, TestLucene50NormsFormat, TestDuelingCodecs, TestBooleanScorer, TestCharsRefBuilder, TestSimpleFSDirectory, MultiCollectorTest, TestLucene50StoredFieldsFormat, TestMaxTermFrequency, TestFieldMaskingSpanQuery, TestOfflineSorter, TestIndexWriterExceptions2, TestSpansAdvanced, TestPerSegmentDeletes, TestSearchWithThreads, TestGraphTokenizers, TestIndexFileDeleter, TestNamedSPILoader, TestLucene50SegmentInfoFormat, TestForTooMuchCloning, TestBufferedIndexInput, TestCodecHoldsOpenFiles, TestEliasFanoSequence, TestFuzzyQuery, TestLockFactory, TestMutableValues, TestSparseFixedBitSet, TestCustomNorms, TestIntsRef, TestSentinelIntSet, TestIndexWriterReader, TestFSTs, TestPackedInts, TestSortedSetSelector, TestFilterSpans, TestMmapDirectory]
   [junit4] Completed on J1 in 1.27s, 38 tests, 1 failure <<< FAILURES!

[...truncated 49 lines...]
BUILD FAILED
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:529: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:477: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/build.xml:61: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/extra-targets.xml:39: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/build.xml:49: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:1363: The following error occurred while executing this line:
/mnt/ssd/jenkins/workspace/Lucene-Solr-5.x-Linux/lucene/common-build.xml:970: There were test failures: 409 suites, 3209 tests, 2 suite-level errors, 2 errors, 30 failures, 1795 ignored (33 assumptions)

Total time: 10 minutes 14 seconds
Build step 'Invoke Ant' marked build as failure
[description-setter] Description set: Java: 32bit/jdk1.9.0-ea-b44 -client -XX:+UseG1GC
Archiving artifacts
Recording test results
Email was triggered for: Failure - Any
Sending email for trigger: Failure - Any